Jump to content
IGNORED

Removers Library Linux Setup Script


BitJag

Recommended Posts

This is a dedicated thread to several scripts I have been maintaining that make setup of RMAC, RLN, JLIBC and RMVLIB tool chain for any ubuntu distribution a nearly one click solution.  I will update this thread when updates occur for those who are interested, and for address any problems you may run into in regards to these scripts.

 

Huge thank you to everyone that has put their time and talents into these tools, and for helping me get to this point that makes these tools more accessible to linux users.

 

There are more details about the script on the github page (linked below), but here is a quick overview of what you need to start and what these scripts give you.

 

You will need to be running a flavor of linux that has access to the Ubuntu repositories, and uses apt to pull packages from the repositories.   I have only tested this on Kubuntu, but it should work with Ubuntu, Xubuntu, Lubuntu, etc... without any issue.  Please let me know if you run into other problems with other distributions that have access to the Ubuntu repositories.

 

The Ubuntu repositories and apt are necessary for the script to install the m68000/gcc cross compiler tools, build-essentials, and a few other applications.  If you are using a distribution that doesn't have access to the Ubuntu repositories, or can't install debian packages, you could potentially bypass these steps in the script by manually downloading and building and installing the cross-mint tools and any other tools that are installed with apt inside these scripts.  Please do not ask help to do this. I will assume that if you venturing outside the boundaries of Ubuntu, you probably know much more about Linux than I do.

 

As mentioned before, the rmvlib_inistall.sh script builds and installs the necessary tools to compile assembly and C code related to the Removers Library.  There is another script called additional_tools_install.sh that will download/build/install other binaries that make Jaguar dev a bit easier.  This includes virtuajaguar, jcp, lz77 and jag-image-converter.  Again more details about these tools on the github page.

 

After you run the install script, there is a small example program inside the Jaguar folder for testing the tool chain to make sure it is working correctly.  Again, see the githup page for more details about getting started.

 

https://github.com/lachoneus/ubuntu-rmvlib-install-scripts

  • Like 3
Link to comment
Share on other sites

Hey, thanks!

 

Also to mention I am working on building a docker environment (based on archlinux) that configures everything.

It is ongoing work but this works reasonably well, even on WSL2

(I still need to adapt to use rmac instead of madmac)

 

Repo is here: https://github.com/theRemovers/jagdev

 

I am also working on translating my Jaguar Image Converter in Python, because it is too difficult to maintain the build with ocamlimages.

  • Like 2
Link to comment
Share on other sites

7 hours ago, SebRmv said:

Hey, thanks!

 

Also to mention I am working on building a docker environment (based on archlinux) that configures everything.

It is ongoing work but this works reasonably well, even on

(I still need to adapt to use rmac instead of madmac)

 

Repo is here: https://github.com/theRemovers/jagdev

 

I am also working on translating my Jaguar Image Converter in Python, because it is too difficult to maintain the build with ocamlimages.

 

This is great news.  Having a more universal option like docker will simplify things considerably.  The Jaguar Image Converter conversion to python will be welcome as well.  I don't know how many hours have disappeared trying to figure out how to build this program from source with ocaml.  Thanks Seb!

Link to comment
Share on other sites

  • 4 weeks later...

Finally got time to work a bit on the Jaguar Image Converter in Python. This should work reasonably well now. I have not ported all the options that were in the ocaml version (dithering/rotation/cut/...)

 

I'd be happy to get feedback on it (as well as on the Docker environment).

 

Cheers

 

Seb

  • Like 3
Link to comment
Share on other sites

Spent a few minutes with the docker image. Followed instructions after getting familiar with the basics of docker.  It seemed to setup the docker container flawlessly, and I had my generic example built in no time after modifying my Makefile's a bit.  This is awesome, I need to try this in Windows now.

 

I am not sure what feedback you are looking for, for the Jaguar Image Converter.  Its great that it is just a python script now, and this will be great in case of future updates. Unfortunately I haven't worked on enough Jag stuff to justify using all the options that the image converter offers.

 

I did do some quick conversions to make sure that the options I typically use still work.  4, 8, 16-bit image depths all converted just fine.  I actually avoid the dithering options in the image converter, as I like my work to look the same before and after it is added to my programs.  I am looking forward to using an up-to-date version of the image converter.

 

Thanks for putting this together Seb!

Link to comment
Share on other sites

On 9/3/2020 at 9:27 AM, SebRmv said:

Hey, thanks!

 

Also to mention I am working on building a docker environment (based on archlinux) that configures everything.

It is ongoing work but this works reasonably well, even on WSL2

(I still need to adapt to use rmac instead of madmac)

 

Repo is here: https://github.com/theRemovers/jagdev

 

I am also working on translating my Jaguar Image Converter in Python, because it is too difficult to maintain the build with ocamlimages.

nice idea @SebRmv

Actually I wonder, how to prepare, based on you GIT, docker file under Windows?

Link to comment
Share on other sites

Actually I meant following code:

cd /jagdev
docker build .

now image is building

 

---EDIT----

 

and it seems failed:

Step 9/49 : RUN   groupadd --gid ${GROUPID} ${GROUPNAME} ||   groupmod --gid ${GROUPID} ${GROUPNAME}
 ---> Running in 0f7363b9d259
groupadd: option '--gid' requires an argument
Usage: groupadd [options] GROUP

Options:
  -f, --force                   exit successfully if the group already exists,
                                and cancel -g if the GID is already used
  -g, --gid GID                 use GID for the new group
  -h, --help                    display this help message and exit
  -K, --key KEY=VALUE           override /etc/login.defs defaults
  -o, --non-unique              allow to create groups with duplicate
                                (non-unique) GID
  -p, --password PASSWORD       use this encrypted password for the new group
  -r, --system                  create a system account
  -R, --root CHROOT_DIR         directory to chroot into
  -P, --prefix PREFIX_DIR       directory prefix

groupmod: option '--gid' requires an argument
Usage: groupmod [options] GROUP

Options:
  -g, --gid GID                 change the group ID to GID
  -h, --help                    display this help message and exit
  -n, --new-name NEW_GROUP      change the name to NEW_GROUP
  -o, --non-unique              allow to use a duplicate (non-unique) GID
  -p, --password PASSWORD       change the password to this (encrypted)
                                PASSWORD
  -R, --root CHROOT_DIR         directory to chroot into
  -P, --prefix PREFIX_DIR       prefix directory where are located the /etc/* files

The command '/bin/sh -c groupadd --gid ${GROUPID} ${GROUPNAME} ||   groupmod --gid ${GROUPID} ${GROUPNAME}' returned a non-zero code: 2
PS C:\Users\cyprian\Downloads\jagdev-master>

 

Edited by Cyprian_K
Link to comment
Share on other sites

  • 2 weeks later...

Is the motivation to encourage building locally just to have the local user/group integrated? Not specific to this container at all (I ran into the same thing putting mine together), but I find this part of docker really annoying. It seems odd that there isn't an easier standard way to import a user and run some related scripts (chown/chmod a bunch of stuff accordingly inside the container) at runtime or something. It makes me wonder if current docker is really ready/built for this interactive type of use case. I've done a little googling, and have only found solutions similar to what Seb has done here. Any docker experts in the room that can point out a good way to handle this? I'd love to be wrong...

Link to comment
Share on other sites

This tweet tells you all you need to know about containers:

 

This is of course the main reason we don't provide linux binaries for rmac and rln: we'd have to assume that the user links against the same version of libc we do - not a great idea.

 

Containers for development is a weird concept: on linux you're pretty much a stone's throw away from installing dev tools and building things from source if something is missing from your setup. On Windows it's a non-issue: binary compatibility goes back decades, so there's a high chance your binary will work (worst case you have to add spefic VC runtime dlls, but those too are standarised). Unsure about macs, but the binaries for rmac/rln we create seem to work fine for people so far.

 

While trying to add the compilers from http://brownbot.mooo.com upstream to http://godbolt.org I had to use containers to test things. It was a horrible experience: tons of MBs of things had to be downloaded and installed somewhere in the disk, and then each test I made the contained had to be spun, run, and had a really bad way to communicate errors when things went wrong. (oh yeah, containers not shutting down and remaining running without any feedback to they user were very fun things too). And of course I remember something along the lines of each time I changed something to the config of the container, it would start downloading the universe again because OMG we can't be sure what we've downloaded already, it might be different!

 

So....... why do people even want this? Why do someone have to spin a mini VM (which is what I understand containers are) in order to just convert a couple of text files into machine language? It doesn't make much sense to me :)

 

On most (all?) of my projects I have 0 things installed. The tools are all contained in the project (usually rmac and gfa32 which also is portable and needs no install). Worst case if I need a C compiler or whatever, I do have a ZIP file with that packed somewhere, ready to be deployed. But even that is not installed, ever.

 

This really makes me curious to ask what people's workflows are. All I'm saying is: keep it simple, people, it's not rocket science!

Link to comment
Share on other sites

10 hours ago, ggn said:

This is of course the main reason we don't provide linux binaries for rmac and rln: we'd have to assume that the user links against the same version of libc we do - not a great idea.

This isn't entirely true.  glibc is actually remarkably good at *backwards* compatibility, so as long as you build against an ancient libc, you can be pretty sure it'll run on anything newer.  If you need libstdc++ or anything else on the other hand, well, good luck.

 

So how do you build against an ancient libc without maintaining crusty build machines?  Containers (or their predecessor, chroots) of course ?

 

In general though, yeah, containers seem kinda neat if you need to spin up 1000 instances of your e-commerce webapp on black friday, but not the greatest for an interactive development environment.  Still, if it makes people less afraid to try rmac/rln/etc., fine by me.

Link to comment
Share on other sites

10 hours ago, cubanismo said:

So how do you build against an ancient libc without maintaining crusty build machines?  Containers (or their predecessor, chroots) of course ?

 

Or just take 5 minutes to install gcc and make and compile it yourself. If you're a dev, and you're already into the linux rabbit hole then installing dev tools shouldn't sound like a daunting task. So, man up and ditch containers I say :D

 

Apologies for the thread derail btw ?

 

(Final note: I just built a version of rmac with statically linked musl using alpine linux. So if that works, we might start distributing linux rmac/rln binaries)

Edited by ggn
Extra thoughts
Link to comment
Share on other sites

1 hour ago, ggn said:

Or just take 5 minutes to install gcc and make and compile it yourself. If you're a dev, and you're already into the linux rabbit hole then installing dev tools shouldn't sound like a daunting task. So, man up and ditch containers I say :D

? To make things even faster, there's also this: https://github.com/wheybags/glibc_version_header . Personally I prefer building rmac/rln from source, but I'm sure someone will appreciate binaries. And yes, apologies as well, I'll try to let the thread return to its regular scheduled programming now.

  • Like 1
Link to comment
Share on other sites

  • 4 weeks later...
  • 1 month later...
On 11/5/2020 at 5:11 PM, Cyprian_K said:

Can someone build this cool container for me?

I made a few attempts under Win/Cygwin, unfortunately to no avail.

Thanks

No time to build it for you, but I did setup a repository for a cygwin version of the setup script for my brother earlier this year to see if it could be done.  We were able to get it working. 

 

Cygwin RMVLIB setup script - https://github.com/lachoneus/windows-cygwin-rmvlib-install-scripts

 

Here is the bad news.  I haven't been updating it along with my linux script.  If you are adventurous though, and you have some experience with bash scripts, you can do a side by side with the linux script(latest version) and the cygwin script(older version) and update the cygwin script, along with the necessary files it may need from the linux script's repository, you should be able to get it working.  I believe rmac has been updated since I put together this script, so some of the lines that manually change rmac source files aren't required anymore.  There may be a few other small things as well.

 

I may approach this again in the future, but my focus is making sure the linux scripts are up to date until I finish Crescent Memories  (someday...).

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...