Build Systems

[snicklin]

I have been learning Go (the language, not the game) a.k.a. Golang on and off for a little while now.

 

Currently I am trying to build up a bit of a library of common functionality that I can reuse.

 

One area I would like to apply this library towards is build systems.

 

I was wondering, what do you use in your build systems? Does anyone have any fancy functionality like processing Excel or CSV files or pulling / updating internet urls or anything even more spectacular? (or interesting)

 

I just want to see what tasks people do in their builds.... maybe I could code something similar for it to make it easier for everyone to use such functionality?
 

Show off as much as you like.....

[popmilo]

That is a good question. So far I've used python scripts to process png files with sprite frames, maps etc.
Sometimes even have information encoded in images like direction, height map etc...

Did plan to use scripts to read output from "Tiled" editor. Think it can output xml, json or similar with all tilemap data with objects that are freely placed around with custom tags.
I know some people here already used it for some cool games years ago. Think it was Xuel & Jose combo, shooter game that was on Abbuc...

 

Other people used php, js, c++ etc... Possibility are endless

[thorfdbg]

4 hours ago, snicklin said:

I was wondering, what do you use in your build systems? Does anyone have any fancy functionality like processing Excel or CSV files or pulling / updating internet urls or anything even more spectacular?

The traditional, old-timer approach: A compiler/assembler/linker (CA65 for the Atari projects), an Editor (emacs) and a Makefile.

 

[phaeron]

Python is useful for operations that are slow or difficult to do in the assembler. For instance, repeating a binary block 8 times is just block*8, and you can use list comprehensions to transform data without having to write loops. (sum(data)-1)%255+1 gives the SIO checksum of a block. Writing an ATR disk image can be done in an assembler, but it's a lot easier to do in a full programming language. I used Python to compile the help data for Altirra Extended BASIC, since it needed transformations for building link-formatted pages from markdown-like syntax, tokenizing the text, and allocating the pages to cartridge banks. As a plus, Python is available on all major platforms and it's relatively easy to write code that works on Windows, Linux, and macOS.

 

It's also a useful language to use for general diagnostics. For instance, [hex(zlib.crc32(data[x:x+1024])) for x in range(0, N, 1024)] will print the CRC32s of each 1K block in binary data -- which is a quick way to tell if you have a bogus ROM dump with duplicated or empty blocks.

 

[Kaj de Vos]

I wrote the build system for the Syllable operating system, in Ruby, and a static website/documentation builder, in REBOL. I intend to port them to my new language. The Syllable build system operates at the package level, so it refers the building of any single package to its chosen build system. It does do a lot of downloading from specified links. For my language, I have also begun writing a sort of make replacement to build the language and single programs. Eventually, the whole language will be available to write build actions. It will be interesting to see what Atari programs need.

[snicklin]

All replies have been good. Even seeing the use of a traditional build system, its good to see what assemblers and editors are in use. I sometimes think that we spend a lot of time here talking about general programming issues and do not talk enough about the programming environments that we are in/using.

 

I see a good variety of systems in use already. Traditionally I have used bat scripts, sometimes Perl for my builds but want to spread my wings further with the power of Go (which is like a modern day 'C' to me). 

 

I like the idea of Atr processing. I have already written the headers for some Atr processing functions but have not filled the. in with code yet. Good to see others are doing this.

 

I suspected the use of Python a lot. It seems a good language for text processing which is what a build system essentially is.

 

One thing I intend to do is some processing of images, so it is good to see Popmilo extracting from images. I will probably do it also, but from Excel spreadsheets. 

 

Kaj de Vos, one thing I have been wondering about is caching. Do you download resources from the internet every time your build runs or do you do any form of checking to see if it is already there? Or do you have some updating flag mechanism to say whether to download or not? I like my builds to be quick and anything internet related will slow my build down.

 

[Kaj de Vos]

I like to use the file system if at all possible, to avoid more complex databases. The Syllable build system just downloads to an appointed folder, and checks if a package or single file is already there, and only downloads it when needed. To refresh a download, or save space, just delete it.

 

Same for unpacking packages. If a source is already there, leave it, otherwise look up the package and unpack it. To throw away an experiment, just delete the source, or go to another work folder. The build system also has some generalised commands for cleaning sources and such operations.

 

There is also support for checking out from versioning systems.

 

When I wrote it, the only build system that generally used the same concepts was Gentoo's Portage. I find most build systems to be overly complex and convoluted.

[ivop]

https://xkcd.com/

[Kaj de Vos]

Somehow I get that one a lot these days. ?Not sure how it's relevant here, though? 

[ivop]

Build systems, like yet another make replacement

 

It was not about your language. Small languages also have their place, and perhaps it grows beyond that

[Kaj de Vos]

I never said I develop anything to be a standard, but people seem to lay that upon it. I develop things when my long-time attempts to use existing stuff don't work out. Often I should have done that years if not decades earlier.

 

In the case of Make, it would be overkill to use if you already have a general-purpose language, and two decades ago it was already inadequate for Syllable. There is no standard I can use without running into problems again.

 

If you mean that all the build systems that were developed after mine shouldn't have been, of course I fully agree. 

[sanny]

"make" is a Turing-complete language. It's not standardized, yes, so my projects require "GNU make". That version of "make" is available for many systems.

[Kaj de Vos]

My platforms include 8-bit machines and WebAssembly. How do I run GMake there?

[sanny]

You have a point ?

 

So, you "build" on these platforms? I was thinking of PC (or similar, e.g. Workstations) based build systems.

[Kaj de Vos]

I certainly want to build in web browsers, and if at all possible, I would like some of that functionality to be available on 8-bit.

[sanny]

Ok. (scratching head)

 

You want to build in web browsers? But on the local machine, not on the web server? I've never thought of that (and don't see a benefit, but that's just me).

[Kaj de Vos]

I avoided web browsers for many years for anything except publishing info pages, what they were designed for. But the force is too strong, people's laziness to install is too strong, and classic program installation has devolved into centralised app stores controlled by not many more than two parties. So if you can't beat them, join them. I still don't want to program in JavaScript, but since a few years, WebAssembly offers a way out. So to do some half-way decent programming in browsers, without installation and without a middleman between me and users, I need to embrace it fully, create a language that fits WebAssembly like a glove, and make it do everything that was traditionally done on workstations.

[popmilo]

Makes sense in 2021.

Best example I can think of is 8bit workshop:
https://8bitworkshop.com/

 

I was skeptic at first, but now I like it. Edit, compile, run anywhere, sync with github, work with local files too...

 

[Kaj de Vos]

Yes, that's an awesome site.

[sanny]

5 hours ago, Kaj de Vos said:

Yes, that's an awesome site.

Yes. Cool.