I have been working on a project called Music Disc Player. The idea is simple on the surface but surprisingly tricky. I recently saw this video:
https://www.youtube.com/watch?v=RnLWLQsh9mw
which allows vanilla Minecraft clients to connect to a server and see 3D objects, crazy right!
No client mods. No resource packs. Just join the server and it works.
The way this works is by leveraging display entities to create something larger. Like the individual pixels (RGB LEDs) on your screen coming together to make an image.
Now there is an internet meme about playing a video called bad apple on everything. For example people do it on the little screen on your refrigerators, in the real world using stop motion with whatever they can find, I even saw one on a pregnancy test (I still have no clue how). This is mostly why I coded this as a bit of a gag lol.
Display entities
The core idea of this project is using display entities to render video frames. Instead of traditional blocks or particles, display entities are more lightweight.
This allows the server to update visuals dynamically without requiring any client-side modifications.
Getting video into Minecraft
Before we start, this project is not very professional and quite messy
Minecraft does not support video playback at all. The first step is to get a video of some form into Minecraft. This was done by breaking down everything into individual frames. These frames have to be sorted somewhere on the server running the game. I then wrote a program to translate it into something the game can understand.
Limitations
- Resolution vs performance Higher resolution looks better but requires more entities or updates. Too many updates per tick causes massive lag.
- Keeping things in sync Video playback needs consistent timing. Minecraft ticks are not designed for smooth media playback, so syncing frames to ticks requires care.
- Audio Unfortunately the best solution I could find is having the user install a resource pack. Minecraft servers do have support for this when someone joins, it can be done with the click of one button. This will allow audio for the videos (currently we are overwriting another music disc), with resource packs we can change how this disc looks also (not implemented). One limitation I could not overcome is the length of the audio must be shorter than the length of the music disc original audio. We cannot touch vanilla Minecraft code and I haven't had any other ideas. Maybe in the future something new will be added that will work but im at a loss for now.
Performance
The biggest issue by far was performance. Rendering video means updating visuals many times per second. While display entites are more efficent then particles we need a truly staggering ammount of them. In Minecraft, it causes massive lag for both the client and server
This project was only possible because of what Mojang is adding and display entities are brand new. This is already pushing it quite far and I'm sure that if the Mojang devs add more cool features like this, it would get to a point of viability.