SDCC PIC16 C compiler?

[intvnut]

Getting this thread back on topic: I personally think there is a lot of value in having an implementation language that is higher level than assembly, but closer to the metal than IntyBASIC. It could serve as a substrate for one or more frameworks, or as a direct implementation language for games.

 

I personally was working on a GCC port, but it's not there and I don't know if it'll ever be there. I at least have a GNU binutils port (including assembler, linker, and disassembler) working.

 

It looks like SDCC can't reasonably support the Intellivision's processor without a lot of fixes throughout its codebase.

 

zezba9000 has an interesting proposal for mapping a dialect of C# to the Intellivision in a way that may elegantly capture the quirkiness of its memory model. I'm skeptical but cautiously hopeful. I think it may be harder to square this circle than it appears. I'd like to hear more and try to flesh out the details.

[+DZ-Jay]

Getting this thread back on topic: I personally think there is a lot of value in having an implementation language that is higher level than assembly, but closer to the metal than IntyBASIC. It could serve as a substrate for one or more frameworks, or as a direct implementation language for games.

 

I personally was working on a GCC port, but it's not there and I don't know if it'll ever be there. I at least have a GNU binutils port (including assembler, linker, and disassembler) working.

 

It looks like SDCC can't reasonably support the Intellivision's processor without a lot of fixes throughout its codebase.

 

zezba9000 has an interesting proposal for mapping a dialect of C# to the Intellivision in a way that may elegantly capture the quirkiness of its memory model. I'm skeptical but cautiously hopeful. I think it may be harder to square this circle than it appears. I'd like to hear more and try to flesh out the details.

 

Would a C# implementation be just a statically compiled language, or an implementation of the CLI for CP-1610?

 

What about a bespoke C dialect with a domain-specific memory model but everything else as is?

[intvnut]

 

I agree with your points there, and that is my preferred model as well. However, I do not see much interest from veterans to provide such frameworks, and the newbies are by definition not skilled to do so.

 

I do what I can when I can with in the programming forum, but unfortunately most people are using IntyBASIC and I am not very steep in it at the moment.

 

However, I would definitely participate in an effort for higher-level abstractions and libraries for any language (for as much as I can). I guess I have just not approached the subject adequately. What I see happening is this:

 

- I want to build a new language

- You know what we really need? Better frameworks.

- Nah, frameworks are for newbies (or frameworks are too limiting).

 

I guess, implicit in my recommendation is my interest in participating on such an endeavor; but I can see how it sounds like a complaint. I'm sorry if that is how it is taken.

 

If we had a community of a 100+ developers, it would make a lot of sense to put a lot of effort into developing a comprehensive framework. And if you realistically think you can grow the Intellivision development community that large, great.

 

We have an Intellivision developer community that grew from less than a dozen (pre-IntyBASIC) to maybe a few dozen (post-IntyBASIC). (I'm speaking in very rough numbers here.) Realistically, what is the size of this community? What is its potential size? Exactly how much leverage do you expect from a particular framework?

 

You can put a lot of effort into defining a framework. The ultimate goal is to get some leverage for that effort: Every hour you spend on the framework saves multiple hours times multiple developers across multiple projects.

 

I don't think the Intellivision community is ever going to be terribly large. IntyBASIC grew the community by nearly an order of magnitude. I don't think a fantastic new framework is going to increase it much further.

 

If I show up with a C compiler or zezba9000 shows up with a C# compiler, that'll be interesting to us. I don't expect it to set the world on fire. For me, I'm interested just in the challenge of seeing a new compiler for this processor.

[intvnut]

Would a C# implementation be just a statically compiled language, or an implementation of the CLI for CP-1610?

 

It sounded like zezba9000's proposal compiles C# down to an intermediate language (IL) which would then get lowered to CP1610 assembly. So, static.

 

 

 

What about a bespoke C dialect with a domain-specific memory model but everything else as is?

 

Well, you get into tricky semantics around what it means to have a char*. It very quickly stops feeling like C.

 

If SDCC supported CHAR_BIT = 16, then you'd probably stop there. It has support for multiple memory pools and so on, and so it would be pretty manageable. SDCC targets other quirky embedded systems and has built in many affordances that would be useful to the Intellivision. Sadly, digging through SDCC's source, CHAR_BIT = 16 is a long row to hoe.

Edited January 5, 2019 by intvnut

[+DZ-Jay]

 

I agree with this. A lang should not be considered or convoluted with frameworks and APIs. Those should be modules you can choose and pick from.

 

 

Also here are some ideas I have for using C# to target Intellivision. From what you have said I think this model I'm suggesting would be highly powerful as it just gives compiler hints rather than forcing types to live in specific memory pools which allows the same game logic code to be shared with Atari 2600/5200, etc.

 

C# allows you to create any custom attribute that can be used on structs, classes, field and methods.

Also FYI a char in C# is 2 bytes already.

 

So take the following C# example (NOTE: structs in C# are the same as C)

using System;

public enum MemoryPoolType
{
  Auto,// auto choose what memory bank to use
  Bank_8bit,// 8 bit memory on Intellivision
  Bank_16bit,// 16 bit memory on Intellivision
  Bank_ROM// ROM cartridge / program memory
}

// IL2X translator looks at this attribute object at compile time when targeting Intellivision ASM output
public class MemoryPoolAttribute : Attribute
{
  public readonly MemoryPoolType type;
  public readonly int poolIndex;

  public MemoryPoolAttribute(MemoryPoolType type, int poolIndex)
  {
    this.type = type;
    this.poolIndex = poolIndex;
  }
}

[MemoryPool(MemoryPoolType.Bank_8bit, 0)]// give compiler error if any type doesn't fix this rule on Intellivision, ignored on single memory pool systems
struct Vec3
{
  public byte x, y, z;
}

// this pattern allows you to have an "object oriented" design and access differnt memory pools from a single object
struct Vec3
{
  [MemoryPool(MemoryPoolType.Bank_8bit, 0)] public byte x, y;// puts in 8bit memory on Intellivision, ignored on single memory pool systems
  [MemoryPool(MemoryPoolType.Bank_16bit, 0)] public short z;// puts in 16bit memory on Intellivision, ignored on single memory pool systems
  // OR
  [MemoryPool(MemoryPoolType.Bank_16bit, 0)] public byte x, y;// puts in 16bit memory on Intellivision as 2 bytes per field, ignored on single memory pool systems
  [MemoryPool(MemoryPoolType.Bank_8bit, 0)] public short z;// puts as 2 bytes in 8bit memory on Intellivision, ignored on single memory pool systems
}


struct Vec3
{
  public byte x, y, z;// would auto choose Bank_8bit on Intellivision
}

struct Vec3
{
  public int x, y, z;// 32bit types are changed to mean 16bit and sizeof(int) would respect this
}

struct Vec3
{
  public int x, y, z;// auto puts into 16 bit memory

  public int Foo(int x, byte abc)// auto puts "x" into 16 bit stack and "abc" into 8 bit stack
  {
    return x + abc;// pulls from 16bit and 8bit stack writing return value to 16bit stack
  }
}

class MyObject
{
  public char prefix;// 2 bytes on Intellivision, 1 byte on Atari 2600
}

Does that make sense? The idea is expandable to more platforms than just Intellivision. Its general purpose.

"poolIndex" is used if there are multiple 16 / 8 bit memory pools to choose from.

 

Heap allocations could also use these attributes. The allocator would just have to keep track of multiple banks for a single class object.

 

I haven't used C# in about 10 years, but from what I see there, it seems you are suggesting that byte and int would automagically map to 8-bit and 16-bit memory storage, but that you could override this with the MemoryPool() hint directive. Is this accurate?

 

 

I'm not familiar with C#, so I'm left wondering how this particular one could possibly work without Vec3 having a top-level structure with pointers to the actual storage:

.

struct Vec3
{
  [MemoryPool(MemoryPoolType.Bank_8bit, 0)] public byte x, y;// puts in 8bit memory on Intellivision, ignored on single memory pool systems
  [MemoryPool(MemoryPoolType.Bank_16bit, 0)] public short z;// puts in 16bit memory on Intellivision, ignored on single memory pool systems
}

.

The normal way C and C++ work is that each non-static member exists at some byte offset relative to the base of the structure, that way you can have a single pointer that describes the location of the structure, and can find the members with a compile-time known byte offset from that pointer.

 

If I have an array of this particular Vec3 in your proposed C# dialect, how do I find a.x, a.y, and a.z? What's the underlying instruction sequence the processor uses to compute the addresses being accessed? (Pseudocode is fine. IL, even, if it's low enough level.)

 

I guess the compiler would have to keep track of each individually via some sort of map. I would also be interested in knowing how that can be solved succinctly. It would be wild!

 

Still, I think the critical point to keep in mind, as intvnut mentioned earlier, is that in the Intellivision 8-bit memory is a separate storage bank from 16-bit memory, and thus would never be contiguous to each other. Abstracting this may mislead programmers into creating unnecessarily complex or expensive structures.

 

-dZ.

[+DZ-Jay]

I guess the compiler would have to keep track of each individually via some sort of map. I would also be interested in knowing how that can be solved succinctly. It would be wild!

 

 

It occurs to me that the compiler could keep track of the base memory for each of the storage type at the definition of the structure, so that the same index could apply to all fields, but compute different absolute addresses. I do something similar in P-Machinery, but of course, at a much more primitive level. I use it for a higher-level "RECORD" structure definition that is similar to "STRUCT," but keeps track of 8-bit and 16-bit memory (and GRAM) separately.

 

-dZ.

[+DZ-Jay]

It sounded like zezba9000's proposal compiles C# down to an intermediate language (IL) which would then get lowered to CP1610 assembly. So, static.

 

OK, that makes sense. So the standard compiler is used, we would just need to provide the bytecode-to-CP1610 translation layer.

 

Well, you get into tricky semantics around what it means to have a char*. It very quickly stops feeling like C.

 

I don't necessarily agree that that's a bad thing, or that it would be so quickly, but that's fair. Still, the concern that you and carlsson brought before are valid: that abstracting some of the innate qualities of the Intellivision platform may not be desirable.

 

If SDCC supported CHAR_BIT = 16, then you'd probably stop there. It has support for multiple memory pools and so on, and so it would be pretty manageable. SDCC targets other quirky embedded systems and has built in many affordances that would be useful to the Intellivision. Sadly, digging through SDCC's source, CHAR_BIT = 16 is a long row to hoe.

 

OK.

[zezba9000]

I'm not familiar with C#, so I'm left wondering how this particular one could possibly work without Vec3 having a top-level structure with pointers to the actual storage:

.

struct Vec3
{
  [MemoryPool(MemoryPoolType.Bank_8bit, 0)] public byte x, y;// puts in 8bit memory on Intellivision, ignored on single memory pool systems
  [MemoryPool(MemoryPoolType.Bank_16bit, 0)] public short z;// puts in 16bit memory on Intellivision, ignored on single memory pool systems
}

.

The normal way C and C++ work is that each non-static member exists at some byte offset relative to the base of the structure, that way you can have a single pointer that describes the location of the structure, and can find the members with a compile-time known byte offset from that pointer.

 

If I have an array of this particular Vec3 in your proposed C# dialect, how do I find a.x, a.y, and a.z? What's the underlying instruction sequence the processor uses to compute the addresses being accessed? (Pseudocode is fine. IL, even, if it's low enough level.)

 

So I didn't think that example all the way through last night (thinking out loud here a little). I was thinking you could create a struct that could ONLY be used on the stack and couldn't be passed around as a method parameter etc. It would just be syntax sugar. However this syntax sugar could just generate two objects that live in each memory pool and when you pass them as method parameters etc it simply passes two objects around. Although a core issue comes around when you want to use this struct in another struct or class. So for now lets assume this syntax and approach isn't practical.

 

Also FYI C# doesn't allow you to access a struct like "a.x". You would get a compiler error as thats considered unsafe code. You can do this BUT ONLY in C# in a "unsafe" code block, method or struct/class. This actually gives you the ability to make clearer rules as what is valid on intellivision in its odd memory system I think.

 

 

Stuff like this should work though (if this makes sense, do you see any holes for the intellivision system?).

[MemoryPool(MemoryPoolType.Bank_16bit)]
struct Vec3
{
  public int x, y, z;// puts into 16 bit memory
  public byte w;// puts into 16 bit memory with 8 bit padding

  public int Foo(int x, byte abc)// auto puts "x" into 16 bit stack and "abc" into 8 bit stack
  {
    return x + abc;// pulls from 16bit and 8bit stack writing return value to 16bit stack
  }

  [return:MemoryPool(MemoryPoolType.Bank_16bit)]
  public int Foo2([MemoryPool(MemoryPoolType.Bank_16bit)] int x, [MemoryPool(MemoryPoolType.Bank_8bit)] byte abc)// explicitly puts "x" into 16 bit stack and "abc" into 8 bit stack
  {
    return x + abc;// pulls from 16bit and 8bit stack writing return value to 16bit stack
  }
}

[MemoryPool(MemoryPoolType.ROM)]
struct StorageObject
{
  public int x;
}

Also here is a quick basic difference between C# and C/C++.

struct MyStruct// structs are always pass by copy and can only live on the heap if contained inside a class
{
  public int x, y, z;// In C# nothing is public by default, even in structs. In C everything is public and C++ structs default to public.

  public MyStruct Foo()// C# is OO so you use normally use methods instead of functions as you would in C++
  {
    // NOTE: "var" is the same as "auto" in C++
    var result = new MyStruct();// the "new" keyword here when used on a struct isn't actually calling an allocator. A struct will always live on the stack when used in a method scope like this.
    MyStruct result = new MyStruct();// same as line above but without "var" syntax sugar keyword.
    return result;
  }

  public void Foo2(ref MyStruct p)// you can pass a struct by ref explicitly if needed (same as "MyStruct& p" in C++)
  {
    p.x = 0;// valid C# code
    p[0].x = 0;// invalid C# code (will get compiler error even though p is a ptr behind the scenes, this is considered unsafe)
  }

  public unsafe void FooUnsafe(MyStruct* p)// you can pass a struct by ptr explicitly if needed (same as "MyStruct* p" in C)
  {
    p->x = 0;// valid unsafe C# code
    p[0]->x = 0;// valid unsafe C# code
  }
}

class MyClass// classes are always pass by reference and always live on the heap (GC objects in short)
{
  public MyStruct s;// inlined directly into MyClasses memory
  public MyClass c;// a pointer / reference to another class

  public Foo(MyStruct someStruct, MyClass someClass)
  {
    someStruct.x = 0;// modifying this value is only modifying a parameter copy that lives on the stack as structs are pass by copy
    someClass.s.x = 0;// this will change "someClasses" value as classes are passing by ref
  }
}

// Now if you wanted to make some "functional" like features you can do stuff like...
static class MyClassExtensions
{
  public static int GetX(this MyClass self)
  {
    return self.s.x;
  }
}

class SomeOtherClass
{
  public void Foo(MyClass c)
  {
    int x = c.GetX();// this method is defined as an extension method external from the MyClass object
  }
}

Hope this helps clear up some basics. For the most part C# is like C++ in general but much more productive as it normally takes a lot less code / files to do the same thing.

 

In either case I'm going to continue work on IL2X and keep the "MemoryPool" attribute idea in mind for intellivision type systems.

Edited January 5, 2019 by zezba9000

[zezba9000]

 

Would a C# implementation be just a statically compiled language, or an implementation of the CLI for CP-1610?

 

What about a bespoke C dialect with a domain-specific memory model but everything else as is?

 

C# is a statically compile language. The reason for using IL via "Mono.Cecil" vs C# directly via "Roslyn" is you get more C# 8.0+ features for free (at least thats the idea). You could even use VisualBasic or F# if you wanted to.

 

So the project i'm working on called IL2X is a .NET IL translator. It converts IL to other native environments. Its works like a .NET decompiler.

Maybe this helps: https://github.com/reignstudios/IL2X

 

In short IL2X could in theory be made to directly translate .NET IL to CP-1610 ASM instructions. Intellivision, Atarti 2600, etc, etc would have a special CoreLib for these platforms. Most Atarti platforms IL2X just translates to CC65 / C89.

 

 

The nice thing about C# vs C, is C# already has all the tools needed to make this special case C# dialect using attributes. C# has a compiler as a service API called Roslyn and Mono.Cecil to analyze IL code as a service. This boils down to you getting the full-blown C# IDE tools on Win, Mac and Lin, such as MonoDevelop, Visual Studio Code or Visual Studio which all have tons of intellisense features etc.

Edited January 5, 2019 by zezba9000

[+DZ-Jay]

C# is a statically compile language. The reason for using IL via "Mono.Cecil" vs C# directly via "Roslyn" is you get more C# 8.0+ features for free (at least thats the idea). You could even use VisualBasic or F# if you wanted to.

 

I asked because my understanding is that it compiles into byte-code for CLI virtual machine (note that I haven't played with C# in several years, so my knowledge about it is rusty). But I see that your intent is that then you get the LI byte-code and translate it statically into the native object code of the target platform.

 

So the project i'm working on called IL2X is a .NET IL translator. It converts IL to other native environments. Its works like a .NET decompiler.

Maybe this helps: https://github.com/reignstudios/IL2X

 

In short IL2X could in theory be made to directly translate .NET IL to CP-1610 ASM instructions. Intellivision, Atarti 2600, etc, etc would have a special CoreLib for these platforms.

Yes, sounds promising. The weirdness of the memory layout is the main issue so far. I'm still not clear how we can overcome that.

 

In your example, you say things like:

// auto puts "x" into 16 bit stack and "abc" into 8 bit stack

So you would expect the CPU to use a 16-bit and an 8-bit stack simultaneously? The 16-bit stack is fine because that's what the CP-1610 currently uses. However, an 8-bit stack will have to be managed manually by the runtime, for there are no provisions for it. Switching the stack pointer to 8-bit stack temporarily is dangerous, for it is used automatically by the built-in vertical blanking (VBLANK) interrupt handler to save the state of the CPU registers, which are 16-bits wide.

 

I guess it could work if you disabled the interrupts temporarily, but that may have an impact on performance or screen refresh.

 

Could you perhaps provide more information on how you think the 8-bit/16-bit split can be handled at the lower level?

 

-dZ.

[zezba9000]

Yes, sounds promising. The weirdness of the memory layout is the main issue so far. I'm still not clear how we can overcome that.

 

In your example, you say things like:

// auto puts "x" into 16 bit stack and "abc" into 8 bit stack

So you would expect the CPU to use a 16-bit and an 8-bit stack simultaneously? The 16-bit stack is fine because that's what the CP-1610 currently uses. However, an 8-bit stack will have to be managed manually by the runtime, for there are no provisions for it. Switching the stack pointer to 8-bit stack temporarily is dangerous, for it is used automatically by the built-in vertical blanking (VBLANK) interrupt handler to save the state of the CPU registers, which are 16-bits wide.

 

I guess it could work if you disabled the interrupts temporarily, but that may have an impact on performance or screen refresh.

 

Could you perhaps provide more information on how you think the 8-bit/16-bit split can be handled at the lower level?

 

-dZ.

 

So correct my thinking if I'm out of line here as working with CPUs at this level isn't something I've ever really needed think about much outside GPU shader programs.

As you know, my thought process was there would be two stacks simultaneously and that means two stack pointers NOT one you switch around (I was thinking both stacks had to be manually managed for each memory bank). I guess I was also thinking the CPU was more like an AVR chip that doesn't even have registers (like some Arduino devices).

 

So does the CP-1610 CPU pull memory from both 8 and 16 bit memory location and store it in registers, or just put memory from 16 bit ones into registers?

 

Lets try to clear up this attribute idea a little more than.

Given this object.

[MemoryPool(MemoryPoolType.Bank_8bit)]
struct Vec3
{...}

void Main()
{
  var v = new Vec3();// this is a stack object but what memory pool do I live in? (I'm marked as living in 8 bit memory)
}

So what options do we have here for where this object should be stored in memory?

1) If we have two stacks each with a separate stack ptr then this "v" instance could live in 8 bit memory. When the stack unwinds, it unwinds both the 8 and 16 bit stacks.

2) If we only have one stack that lives in 16 bit memory does this "v" instance get created on the 16 bit stack with memory padding? "MemoryPool" in this case is just a compiler hint.

3) other options?

Edited January 5, 2019 by zezba9000

[+DZ-Jay]

So correct my thinking if I'm out of line here as working with CPUs at this level isn't something I've ever really needed think about much outside GPU shader programs.

As you know, my thought process was there would be two stacks simultaneously and that means two stack pointers NOT one you switch around (I was thinking both stacks had to be manually managed for each memory bank). I guess I was also thinking the CPU was more like an AVR chip that doesn't even have registers (like some Arduino devices).

 

Ah, I see. Not really. The CP-1610 is a full-featured microprocessor, with an instruction set similar to the PDP-10. It has 8 16-bit registers: a data accumulator, 3 general purpose indexing registers, 2 more general purpose indexing registers that autoincrement when accessing memory, a stack pointer which doubles as another autoincrement indexing register, and a program counter.

 

So does the CP-1610 CPU pull memory from both 8 and 16 bit memory location and store it in registers, or just put memory from 16 bit ones into registers?

 

It pulls memory from 8-bit and 16-bit storage into registers. It is not byte-addressable, though, only word-addressable, so all access is in reality 16-bits. For 8-bit memory, it just zeros out the high order byte, but the target register still treats it as a 16-bit value.

 

That's why intvnut keeps talking about changing the "char" low-level basic structure to 16-bit, because the CP-1610 is a 16-bit processor that cannot access individual bytes.

 

There are provisions to reconstruct 16-bit values from 8-bit or 10-bit ones by modifying an instruction to read twice with an autoincrement register. This goes back to the time when ROMs were primarily 10-bits wide (something that the CP-1610 supports as well), and 16-bit addresses were split into two "DECLES," which is the Intellivision jargon for a 10-bit byte.

 

Lets try to clear up this attribute idea a little more than.

Given this object.

[MemoryPool(MemoryPoolType.Bank_8bit)]
struct Vec3
{...}

void Main()
{
  var v = new Vec3();// this is a stack object but what memory pool do I live in? (I'm marked as living in 8 bit memory)
}

So what options do we have here for where this object should be stored in memory?

 

1) If we have two stacks each with a separate stack ptr then this "v" instance could live in 8 bit memory. When the stack unwinds, it unwinds both the 8 and 16 bit stacks.

It would have to be a logical structure kept at runtime. That, you can imagine, has significant overhead.

 

2) If we only have one stack that lives in 16 bit memory does this "v" instance get created on the 16 bit stack with memory padding? "MemoryPool" in this case is just a compiler hint.

 

Padding 8-bit in 16-bit memory would work, but it means we are essentially forgoing the use of 8-bit memory, no? That may be fine for RAM variables and such, but there are peripherals and other devices which are 8-bits wide, including the graphics RAM bus.

 

It would be ideal to have a single memory model that applies to all of them.

 

3) other options?

It seems that there needs to be a core separation between both types of memory, and that this cannot (or should not) be abstracted to avoid misleading the programmer.

 

Two logical stacks could be used, but I can't imagine the management overhead being cheap.

 

I'm curious to know what intv has to say. He's the resident expert on Intellivision hardware and software.

 

dZ.

[zezba9000]

The Graphics RAM (GRAM) at $3800 - $39FF is 8-bit and only accessible during vertical retrace or when the display is blanked

 

All of it is addressed in 16-bit steps. The 8-bit RAM just zeros out bits 15:8.

 

 

Looking back to this. Is it safe to say that the built in 8 bit memory on the Intelvision is primarily used for GPU / graphics ram? If so did a separate graphics chip read this memory for blanking? Just wondering.

 

Could you perhaps provide more information on how you think the 8-bit/16-bit split can be handled at the lower level?

 

What if any memory that was stored in 8 bit memory could only be done via heap allocations? Or any other external memory pool for that matter. Then there is only one 16 bit stack memory pool, which removes the need for any complexity here.

 

So in C# terms this could be enforced by using a class (as its a ptr) and setting its MemoryPool to 8 bit memory as it would enforce that compiler rule.

[MemoryPool(MemoryPoolType.Bank_8bit)]
class MyGraphicsData// if this was a struct you would get a compiler error
{
  public byte r, g, b;
  public int x, y, z;// two bytes
}

class NormalObject// if MemoryPool not set, assume main 16 bit pool
{
  public byte r, g, b;// 16 bit from padding but used as 8 bit
  public int x, y, z;// 16 bit int
}

struct SomeStruct
{...}

void Main()
{
  var graphics = new MyGraphicsData();// heap allocated in 8 bit / graphics memory pool
  var normalObject = new NormalObject();// heap allocated in 16 bit memory pool
  var someStruct = new SomeStruct();// stack allocated in 16 bit memory as all stack objects would be
}

// Stuff like ROM memory access could be done with an attribute like:
[MemoryPool(MemoryPoolType.ROM, 0xFFFF)]// addressed by memory location?
static class MyReadOnlyData
{
 public readonly static int x, y, z;// only one readonly instance possible (thus static)
}

Keep in mind heap allocation in IL2X micro GC system would only take one extra byte. The GC also de-fragments memory.

Edited January 6, 2019 by zezba9000

[zezba9000]

That's why intvnut keeps talking about changing the "char" low-level basic structure to 16-bit, because the CP-1610 is a 16-bit processor that cannot access individual bytes.

Well in C# a char is 2 bytes

 

Take a look at my last post, I revised the idea from stuff you guys have posted. I think it may solve the issues you brought up.

[intvnut]

 

Well, you get into tricky semantics around what it means to have a char*. It very quickly stops feeling like C.

 

 

 

I don't necessarily agree that that's a bad thing, or that it would be so quickly, but that's fair.

 

Regarding my "quickly" comment: The C memory model is that RAM is an array of char, and other objects are composed of sequences of char. This is what allows memcpy and malloc to work, and gives sizeof() meaning. Pointers to arbitrary objects are convertible to char* and back. If you mess with the semantics of char and char*, you very quickly have a new language.

 

That makes it hard to piggyback on existing C compilers, which would be an explicit goal for me if I were to complete a C compiler backend. When you piggyback on existing compilers, you inherit all of their semantic checking on the front end, and optimizations in the middle end.

 

If you create a new language, you are now on the hook for a new frontend, at the very least, or extensions to the C front end to correctly interpret the semantics of this new dialect. And, if you've mucked with char* too much, you probably lose a significant fraction of the standard library.

 

I hope that clarifies my comment.

[intvnut]

I'm just catching up on this thread.

 

Looking back to this. Is it safe to say that the built in 8 bit memory on the Intelvision is primarily used for GPU / graphics ram? If so did a separate graphics chip read this memory for blanking? Just wondering.

 

There are up to 3 8-bit memory pools.

• Scratch RAM at $100 - $1EF. This is intended for game variables and is accessible at all times. 240 total bytes.
• ECS RAM at $4000 - $47FF. This is only available if the ECS is attached. This is where ECS BASIC stored your BASIC programs and variables.
• 
  
• Graphics RAM at $3800 - $39FF. This is more like GPU memory and is primarily owned by the display controller chip (STIC). Games mostly write to this memory to update graphic tile pictures. Reading isn't common outside of certain types of initialization steps.
• 
  

Scratch RAM is perhaps the most important to consider as it's in each Intellivision, and represents just over half of the available program storage for variables in a standard, unexpanded Intellivision.

 

The rest is 16-bit System RAM at $2F0 - $35F, which gives you 112 16-bit words of memory (224 bytes). Note that the processor stack must also reside in 16-bit memory in order for interrupts to function correctly.

 

 

Ah, I see. Not really. The CP-1610 is a full-featured microprocessor, with an instruction set similar to the PDP-10. It has 8 16-bit registers: a data accumulator, 3 general purpose indexing registers, 2 more general purpose indexing registers that autoincrement when accessing memory, a stack pointer which doubles as another autoincrement indexing register, and a program counter.

 

Minor quibble: It's based on the PDP-11, not PDP-10, heavily chopped down.

 

One major feature the CP-1610 lacks is indexed addressing. This really adds a lot of pain if you want to do a lot of work with stack frames, since you can't just say @SP[offset], or @R3[offset], or so on. Some compilation models really want to be able to set up stack frames and then use offsets relative to the stack to access local variables, etc. That will be expensive on the Intellivision.

 

I mention this as I see many uses of the word "stack" in the discussion. Stack accesses on the Intellivision (even for a logical stack) are pricey, and so you really want a compilation model that allows register allocation for efficiency. Perhaps stack is just a logical construct here, and it's up to the code generator back end to turn "stack" references back into virtual registers that then get allocated to machine registers for improved efficiency. (Much like a JIT would do for Java.)

 

 

What if any memory that was stored in 8 bit memory could only be done via heap allocations? Or any other external memory pool for that matter. Then there is only one 16 bit stack memory pool, which removes the need for any complexity here.

 

I would strongly consider a model in that direction: Make allocations of 8-bit quantities in 8-bit memory explicit and special. Because CP-1610 is a 16-bit word oriented machine, 8-bit memory looks weird: It's memory that zeros out bits 15:8. It doesn't fit the "memory is a bag of bytes" model that most computers have.

 

You really want two different kinds of pointers, and rules against converting between the two (except possibly with explicit pain): Pointers to 8-bit memory and pointers to 16-bit memory. The rules for accessing each are a bit different. You can store a 16-bit value in 8-bit memory across two locations, and can read it somewhat cheaply using the "SDBD" prefix in CP-1610 assembly.

 

 

(Programming note: In C, char == byte == CHAR_BIT bits, and that may be something other than 8 bits. In C#, it sounds like byte is always 8 bits, and char is always 16 bits. This can make conversation very confusing if we don't say what we mean. If anything I say is unclear, let me know. I'll try to be as explicit as possible.)

 

 

Also FYI C# doesn't allow you to access a struct like "a.x". You would get a compiler error as thats considered unsafe code. You can do this BUT ONLY in C# in a "unsafe" code block, method or struct/class. This actually gives you the ability to make clearer rules as what is valid on intellivision in its odd memory system I think.

 

Are you saying C# doesn't allow you to allocate an array of structures? I was referring to something like:

 

.

Vec3 a[16];

for (i = 0; i < 16; i++) {
   // do stuff with a[i].x etc. here
}

.

That's a pretty common paradigm. For example, declaring an array of objects to track my MOBs / sprites, and then using array indexing to work with them directly.

Edited January 6, 2019 by intvnut

[zezba9000]

There are up to 3 8-bit memory pools.

• Scratch RAM at $100 - $1EF. This is intended for game variables and is accessible at all times. 240 total bytes.
• ECS RAM at $4000 - $47FF. This is only available if the ECS is attached. This is where ECS BASIC stored your BASIC programs and variables.
• Graphics RAM at $3800 - $39FF. This is more like GPU memory and is primarily owned by the display controller chip (STIC). Games mostly write to this memory to update graphic tile pictures. Reading isn't common outside of certain types of initialization steps.

Scratch RAM is perhaps the most important to consider as it's in each Intellivision, and represents just over half of the available program storage for variables in a standard, unexpanded Intellivision.

 

The rest is 16-bit System RAM at $2F0 - $35F, which gives you 112 16-bit words of memory (224 bytes). Note that the processor stack must also reside in 16-bit memory in order for interrupts to function correctly.

Ok so knowing this I would say all heap allocations default to the "scratch" memory pool with the option to heap allocate in the processor memory pool of course but sounds like that should be left for stack memory at default.

 

You really want two different kinds of pointers, and rules against converting between the two (except possibly with explicit pain): Pointers to 8-bit memory and pointers to 16-bit memory. The rules for accessing each are a bit different. You can store a 16-bit value in 8-bit memory across two locations, and can read it somewhat cheaply using the "SDBD" prefix in CP-1610 assembly.

 

With the attribute model in C# the compiler will know what kind of pointer type to use if needed. If wanted you can add a special case field attribute to specify how a 16 bit number is divided in memory but this sounds very rare.

 

In C#, it sounds like byte is always 8 bits, and char is always 16 bits. This can make conversation very confusing if we don't say what we mean.

 

Yes on most all platforms it would be BUT .NET is designed so this can change for specific platforms if needed. On Atari 2600 for example I will probably make char 8 bit even though its 16 bit on a desktop to handle the small memory limits better. On Intellivision C#'s byte = 1 and char = 2 fits perfectly. Byte is conceptually still used a 1 byte and sizeof(byte) still returns 1, its just padded in 16 bit memory.

 

Are you saying C# doesn't allow you to allocate an array of structures? I was referring to something like:

 

No C# allows you to allocate an array of anything. I thought you were talking about a C style ptr could be accessed like an array in C when its not. Which isn't allowed in C# without using "unsafe" code as shown in my example.

[intvnut]

 

Given this object.

[MemoryPool(MemoryPoolType.Bank_8bit)]
struct Vec3
{...}

void Main()
{
  var v = new Vec3();// this is a stack object but what memory pool do I live in? (I'm marked as living in 8 bit memory)
}

So what options do we have here for where this object should be stored in memory?

1) If we have two stacks each with a separate stack ptr then this "v" instance could live in 8 bit memory. When the stack unwinds, it unwinds both the 8 and 16 bit stacks.

2) If we only have one stack that lives in 16 bit memory does this "v" instance get created on the 16 bit stack with memory padding? "MemoryPool" in this case is just a compiler hint.

3) other options?

 

I'm a little confused on what constitutes a stack in your model and explanations. For me, a stack is a recursively nested structure that potentially grows with each nested call frame. Your new statement above seems like it should allocate from a heap, and not have any particular nesting semantics.

 

In the hardware, there is, at a minimum, the processor stack, represented by the processor register R6 (also called SP). This stack pointer holds function return addresses, and in the case of interrupts, the contents of all CPU registers at the point the interrupt was taken.

 

We can create however many software stacks we like, if it makes sense to do so. Reading your descriptions more carefully, though, it sounds like what you're really describing are heaps—pools of memory we can draw allocations from, and then the allocation lifetimes are either explicitly managed, or reaped by a GC.

 

Our pools of memory are so small, that typical games rely on static allocations to avoid the overhead of a heap manager or GC to manage the memory. Or, with games that have multiple phases, each phase has a static allocation that's separate of the other phases, and lifetimes are managed by game-phase boundaries. Again, explicit memory management.

 

In C, you might represent static allocations as global variables. Or, for the multi-phase model, multiple structs overlaid in a union. I don't know what the intended memory management model is in C#.

 

My suggestion would be to guide toward a more static model for memory allocation if at all possible.

[intvnut]

No C# allows you to allocate an array of anything. I thought you were talking about a C style ptr could be accessed like an array in C when its not. Which isn't allowed in C# without using "unsafe" code as shown in my example.

 

In the C model, an array of Vec3 decays to a pointer to the first array element in an expression. (And that includes the expression a.) Structures are stored in contiguous memory, and array elements are stored consecutively in memory. Thus, array indexing in C boils down to scaling the index, adding it to the base pointer, and then adding the offset of the member. The code generation model for something like a.y is fairly straightforward in the C model of memory. The underlying arithmetic the processor needs to perform looks approximately like a + i*sizeof(struct Foo) + offsetof(Foo, y).

 

When you were suggesting a structure that had some fields in 8-bit memory and some fields in 16-bit memory, I wondered how that might work, as you now had a structure whose fields were not stored in contiguous memory. (I'm referring to the version of Vec3 that had x and y tagged for 8-bit memory, and z tagged for 16-bit memory.)

 

The idea that the processor could access a particular field of the structure by adding an offset to the pointer to the base of the structure breaks down. And, if I declare an array of such structures, the problem compounds. I now need to know how to translate a.x and a.z with different scale factors and different base addresses for each, given that the x and y fields would be in 8-bit memory and the z fields would be in 16-bit memory. It seemed like, to make this work, you'd need an additional level of indirection under the hood to say where all of Vec3's 8-bit values were vs. its 16-bit values.

 

Do you follow what I'm getting at?

 

I don't think you can easily arrive at a model where you can define a heterogeneous structure, with some fields in 8-bit RAM and some in 16-bit RAM, that doesn't have an additional level of (hidden) indirection, unless all allocations are static at compile time.

[+DZ-Jay]

Joe, I think he actually means stack. He mentioned that in C#, structure are allocated in the stack, as a frame. Also, call-by-value allocates arguments on the stack as well.

 

For what it's worth, I'm with intvnut on the static allocation of memory, the explicit separate pointers for each storage type, and register allocation instead of software stacks, which can be expensive.

 

dZ.

[intvnut]

Joe, I think he actually means stack. He mentioned that in C#, structure are allocated in the stack, as a frame. Also, call-by-value allocates arguments on the stack as well.

 

I'm trying to square the call to new with stack. The local variable holding the reference to the allocated storage is on the stack, but the actual allocated object should come from a heap.

 

Also, if objects on the stack can survive outside their call frame (say, by a lexical closure), then the stack itself doesn't really nest. Or, the stack only retains references and the objects themselves are on a more general heap.

Edited January 6, 2019 by intvnut

[zezba9000]

 

I'm a little confused on what constitutes a stack in your model and explanations. For me, a stack is a recursively nested structure that potentially grows with each nested call frame. Your new statement above seems like it should allocate from a heap, and not have any particular nesting semantics.

 

A stack is just as you describe. In my generic C# code example I tried to explain C# just makes it look likes its a heap allocation when its not if you come from a C++ background.

Notice my code example in my post above with line:

var result = new MyStruct();// the "new" keyword here when used on a struct isn't actually calling an allocator. A struct will always live on the stack when used in a method scope like this.

New in C# is just stating its new memory which can be on the stack (struct) or heap (class). Structs are always allocated on the stack and a Class is always allocated on the heap. Its just a syntax thing but done this way because C# has a much more powerful object initializer than C/C++

"auto result = MyStruct()" in C is the same as "var result = new MyStruct()" in C#.

 

Reading your descriptions more carefully, though, it sounds like what you're really describing are heaps—pools of memory we can draw allocations from, and then the allocation lifetimes are either explicitly managed, or reaped by a GC.

 

Yes exactly. I would focus on this example below as it solves the issues we're talking about. Look at that example above it came from in my earlier post.

var graphics = new MyGraphicsData();// heap allocated in 8 bit / graphics memory pool

As I said before IL2X would have a micro GC for heap allocations that only requires extra 1 byte per allocation. This GC design also uses a defrager so the memory aligns much like stack memory would. This approch would be acceptable for heap allocations.

 

Our pools of memory are so small, that typical games rely on static allocations to avoid the overhead of a heap manager or GC to manage the memory.

 

In C# if you wanted to statically allocate objects we could do this as well and choose what memory pool it goes in.

Example:

unsafe struct MyStruct
{
 public fixed byte myFixedBuff[16];// C style buffer expressed in C#
}

[MemoryPool(MemoryPoolType.Bank_16bit)]
static class MyClass
{
  public static MyStruct myStruct;// would be allocated in the 16bit memory pool (doesn't use GC)
}

Edited January 6, 2019 by zezba9000

[intvnut]

 

I'm trying to square the call to new with stack. The local variable holding the reference to the allocated storage is on the stack, but the actual allocated object should come from a heap.

 

Also, if objects on the stack can survive outside their call frame (say, by a lexical closure), then the stack itself doesn't really nest. Or, the stack only retains references and the objects themselves are on a more general heap.

 

Let me expand on what I mean by "lexical closure" with an example in Perl.

.

sub makeCounter {
    my $counter = 0;
    return sub { return $counter++; }
}

.

The variable $counter was declared locally to makeCounter. However, the anonymous sub (called a lambda expression in many languages) captures a reference to $counter, and so $counter continues to exist after makeCounter returns.

 

In a purely nested stack environment, that wouldn't work. So, Perl's compilation model has to understand that references to objects can persist beyond the life of a stack frame. One way to do that is to have the stack frame only hold references, and the objects themselves come from a heap. GC and reference counting handles the rest.

 

The call to new seems consistent with a model where objects come from the heap, and local variables on the stack only hold references.

[+DZ-Jay]

 

In the C model, an array of Vec3 decays to a pointer to the first array element in an expression. (And that includes the expression a.) Structures are stored in contiguous memory, and array elements are stored consecutively in memory. Thus, array indexing in C boils down to scaling the index, adding it to the base pointer, and then adding the offset of the member. The code generation model for something like a.y is fairly straightforward in the C model of memory. The underlying arithmetic the processor needs to perform looks approximately like a + i*sizeof(struct Foo) + offsetof(Foo, y).

 

When you were suggesting a structure that had some fields in 8-bit memory and some fields in 16-bit memory, I wondered how that might work, as you now had a structure whose fields were not stored in contiguous memory. (I'm referring to the version of Vec3 that had x and y tagged for 8-bit memory, and z tagged for 16-bit memory.)

 

The idea that the processor could access a particular field of the structure by adding an offset to the pointer to the base of the structure breaks down. And, if I declare an array of such structures, the problem compounds. I now need to know how to translate a.x and a.z with different scale factors and different base addresses for each, given that the x and y fields would be in 8-bit memory and the z fields would be in 16-bit memory. It seemed like, to make this work, you'd need an additional level of indirection under the hood to say where all of Vec3's 8-bit values were vs. its 16-bit values.

 

Do you follow what I'm getting at?

 

I don't think you can easily arrive at a model where you can define a heterogeneous structure, with some fields in 8-bit RAM and some in 16-bit RAM, that doesn't have an additional level of (hidden) indirection, unless all allocations are static at compile time.

You could, if you track the base addresses of each storage type separately for the structure. However, this would be exceedingly expensive if done at runtime. Statical allocation simplifies it, but the compiler would have to still track each base individually.

 

I do a similar thing in P-Machinery, with additional metadata associated with the structure that tracks the base of each storage type. In my framework, the model is that memory of the same type defined in a record structure is allocated contiguously.

 

However, I do not know how practical this is in a C# compiler or LI.

 

dZ.

[+DZ-Jay]

Ah, I think I get what's going on. C# is Heljsberg's baby, so structs work like Delphi's RECORDs, which are always allocated on the stack and passed by value, unless allocated from a typed record pointer.