Full Motion Video concept

[+MarcoJ]

 

Hi All,

 

An implementation of a FMV (full motion video) concept for the Atari 2600 has been developed, based on the Pitkat kernel. The resolution is 64x32 pixels with 64 colour cells, running at 30 fps. 

 

** This was demonstrated on the Zero Page Homebrew Show on 2021-03-19. ** 

 

Demonstration Files

 

Here are 3 demonstration videos stored on 16 KB ROMs.

 

1. Rally. This shows an airborne rally car land onto the ground. The texture is created with thresholding. The fast movement on the car and scenery helps the viewer believe the animation. The detail of the scene is not as important as the movement. Colour plays a small role in conveying the sky, mountainside and the road.

 

2. Sprintcat. It’s a video of a cat sprinting on grass. The texture is also created with thresholding. The large size and fluid movement of the cat encoded into the black texturing helps the scene be believable. The texture is everything in this video; colour doesn’t play a significant role.

 

3. Simpsons. A video of the Simpsons sitting on their couch. This video uses edge detection rather than thresholding to create solid black lines around the objects, as opposed to relying on large moving black objects used in the first two videos. It uses colour to the highest degree out of all the demos. However, it struggles with an overload of detail and colour in the scene. *EDIT 27-3-21 Added a new version of Simpsons where Horizontal colour resolution is doubled by use of a flicker technique i'm dubbing "Venetian Comb". Looks best on a CRT and in Stella with "Phosphor TV Effect" enabled, 60% or better. If you pause Stella you can see how it works.

 

 

A detailed explanation is below. Let us know what you think. Cheers, Marco Johannes

RALLY E716k.bin SIMPSONS E716k.bin SPRINTCAT E716k.bin

SIMPSONS E716k V2 - Venetian Comb.bin

E716K Video Experiment test rom CAT V1.bin

[+MarcoJ]

2-4-21 - A more developed concept called the "Moviecart" has just been made publicly aware as of 30-3-21. Check it out, it's incredible.  

 

 

 

Introduction

- It was of interest to know if the Pitkat kernel could be used to play colour video.

- The original need of the video concept was to test a glitch Pitkat caused on James@ZPH console.
- Colour video has been done before in 2003! Dancing Baby by Andrew Davie. Check it out below. 

- A fully developed video concept was released to public knowledge 30-3-21 called the Moviecart. See above.

- Monochrome video has been done before - Bad Apple demo.

 

- My implementation allows a 64 x 32 pixel monochrome texture with an 8 x 8 colour cell matrix, with 1 of 128 colours available for each colour cell.

- Can change colour and texture in one frame, allowing 60 frames per second video. It can be slowed down too to 30 fps, 20fps 15fps, etc.

- Uses a newly developed kernel for Pitkat Revision 3 which exploits an undocumented TIA glitch which helps achieve drawing of the screen without vertical and horizontal gaps between tiles.

- To produce the video information during compile time, video data is encoded in 2 processes to produce texture and colour data.

- The process uses tools such as ImageMagick, GIMP, Binary to Hex converter and Excel.

- So far, grayscale thesholding and edge detecting have been used to create the monochrome textures.

- Creating the colour data involves shrinking each frame to an 8x8 image and extracting the colour. The shrinking process averages the colour over the whole colour cell area into a single colour that’s used for the colour cell.

- With E7 16k bank switching, it can fit 30 frames in the ROM. This is equivalent to 1 second of video at 30 frames per second. This is obviously very limiting.

- Even if the ROM was expanded to be 512kB, this would still be only 32 seconds of footage.

- The bigger picture of the concept would be to able to stream video from an SD card. Such concepts have been implemented on early computers such as the ZX Spectrum with aftermarket SD card interfaces.

- The current video bitrate is 10 kBit/s, or 36 Megabytes per hour. With a 16GB card, it would be possible to store 444 hours of footage.

- There is currently no video compression whatsoever. Textures and colours are stored in a Raw format in order to provide maximum throughput. Given the huge capacity of SD cards, this isn’t a problem.

 

Overall Impressions

 

- Movement is the most important requirement for footage to be believable.

- Large, fluidly moving objects contrasting against the background seem to work the best.

- Scenes with large patches of similar colours work better than scenes with many contrasting coloured objects.

- It struggles with static and detailed scenes as they are lost in the low resolution graphics and limited colour cells, especially horizontally.

- Scenes containing a lot of black are also problematic as the colour cell averaging process tends to pick black as the dominant colour and this causes large sections to become invisible. An example of problematic footage would be a space scene or night footage.

 

 

Future Goals

 

- The next goal was to have sound. 8 kHz, 4 bit digitized sound could accompany the video. The concept is yet to be proven. It is estimated that the video bitrate would double to 20 kBit per second. (72 MB/hour).

- With more cart RAM, it would be possible to host a second kernel to fill the interlaced lines using 30 Hz flicker.

- The ultimate goal was to stream video off an SD card. This has already been achieved now by the Moviecart. Thus, this concept will not develop further.

- It should be possible to increase the resolution of the video from 64 to 96 pixels with Bus Stuffing. This scheme should be capable of updating texture and colour cells for a complete Medium space triple copied players venetian blinds kernel(12 x 8 pixels horizontally). Conventional banking schemes are unable to match this throughput. Perhaps this will eventuate one day.

Edited April 2, 2021 by MarcoJ
Update

[+splendidnut]

This is really cool! Nice job.

[+Andrew Davie]

Actually, dancing baby was colour not monochrome.

 

[+MarcoJ]

18 minutes ago, Andrew Davie said:

Actually, dancing baby was colour not monochrome.

 

Whoa...Did you end up making a colour version? I recall it from a while ago now, seeing a cascading colour on the sides and B/W in the centre.  Have I missed this breakthrough?

Edited March 20, 2021 by MarcoJ

[+Andrew Davie]

10 minutes ago, MarcoJ said:

Whoa...Did you end up making a colour version? I recall it from a while ago now, seeing a cascading colour on the sides and B/W in the centre.  Have I missed this breakthrough?

Are we even talking about the same thing, I wonder?

 

 

ohno!colour.bin

[+MarcoJ]

2 hours ago, Andrew Davie said:

Are we even talking about the same thing, I wonder?

 

 

ohno!colour.bin 32 kB · 1 download

Oh wow! That is awesome. And 2003! Sorry for getting it wrong Andrew, I have updated the spiel above. There is someone on youtube putting up videos of Atari demos. I think it must have been an old alpha?

 

 

Edited March 20, 2021 by MarcoJ
URL cache image

[+Andrew Davie]

Just now, MarcoJ said:

Oh wow! That is awesome. And 2003! Sorry for getting it wrong Andrew, I have updated the spiel above. There is someone on youtube putting up videos of Atari demos. I think it must have been an old alpha?

 

Ah yes. So long ago I didn't even remember it.

 

[Prizrak]

Very impressive@MarcoJ

Sent from my SM-N960U using Tapatalk

[+MarcoJ]

27-3-21 Added a new version of Simpsons where Horizontal colour resolution is doubled by use of a flicker technique i'm dubbing "Venetian Comb". Looks best on a CRT and in Stella with "Phosphor TV Effect" enabled, 60% or better. If you pause Stella you can see how it works. See top of forum for download. Cheers - Mark.

[Bomberman94]

The 3 demos often crashed with my light sixer Atari 2600 PAL using Harmony cart. Running fine with my PlusCart with the same console ?‍♂️

[+MarcoJ]

4 minutes ago, Bomberman94 said:

The 3 demos often crashed with my light sixer Atari 2600 PAL using Harmony cart. Running fine with my PlusCart with the same console ?‍♂️

Wow, cool. Does it crash immediately on load, or it runs for a while? 

[Bomberman94]

1 minute ago, MarcoJ said:

Wow, cool. Does it crash immediately on load, or it runs for a while? 

It crashes immediately ?

[+MarcoJ]

2 minutes ago, Bomberman94 said:

It crashes immediately ?

Interesting indeed. Thanks for the feedback. This happened on 2 other user's machines with Pitkat. I'm hoping to understand the reason one day.

[Bomberman94]

58 minutes ago, MarcoJ said:

Interesting indeed. Thanks for the feedback. This happened on 2 other user's machines with Pitkat. I'm hoping to understand the reason one day.

Yes, same problem with Pitkat - first started (after power on) it works but after a reset or short power off it won’t run. Won‘t run means black screen or stripes and sometimes an annoying noise. 

[+MarcoJ]

44 minutes ago, Bomberman94 said:

Yes, same problem with Pitkat - first started (after power on) it works but after a reset or short power off it won’t run. Won‘t run means black screen or stripes and sometimes an annoying noise. 

Indeed. this is the exact same reported fault. If you see on ZPH's historical show at 26:50, the same thing happens.

Interestingly, when they switched consoles several weeks later the problem stopped. You are the 3rd user to report this issue on their console.

[+MarcoJ]

Wow! I've just heard the news that a full motion video implementation with sound has been fully developed by a developer called Iodefmode  aka @rbairos. It will run full length movies on a custom cartridge called a "Moviecart" on the Atari 2600. It is going to make its debut on the @ZeroPage Homebrew show on Tue Mar 30 6PM PT | 9PM ET | 2AM GMT. This is exciting news!

Edited March 31, 2021 by MarcoJ
identity confirmed.

[+rbairos]

On 3/29/2021 at 8:11 PM, MarcoJ said:

Wow! I've just heard the news that a full motion video implementation with sound has been fully developed by a developer called Iodefmode  aka @rbairos. It will run full length movies on a custom cartridge called a "Moviecart" on the Atari 2600. It is going to make its debut on the @ZeroPage Homebrew show on Tue Mar 30 6PM PT | 9PM ET | 2AM GMT. This is exciting news!

Thanks very much @MarcoJ . Also impressed with what you were able to do without any sort of hardware chicanery! Very cool