Home       About Me       Contact       Downloads       Part 31    Part 33   

Part 32: Mining Minecraft

September 10, 2011

Back in Part 29, I mentioned that I might want to write some kind of import tool to pull data from Minecraft. It wasn't very high on my list. The more I thought about it though, the better this sounded.

My game will appeal to the same people who like Minecraft. If they've put a lot of time into building something there, they won't want to start all over in my world. Plus there are a lot of great buildings already out there. A way to give some initial content to my game world would be to import it all from some Minecraft server.

There are drawbacks. Importing Minecraft buildings means my world will have to handle all of the Minecraft brick types, in addition to whatever I add. The import tool will have to be updated whenever Notch decides to change his format. It puts some Minecraft-dependent code into my game, leading to version mismatches. But the value of all those existing buildings outweighs the problems.

Back in Part 7, I had written code to unpack the Minecraft Alpha file format and convert it to my Octree format. It wasn't exactly interactive! I just ran a script over the entire world to decompress each chunk with gzip. Then read the uncompressed data into a huge array in memory. Finally, I wrote it all out again in my format. Crafty read only my save files.

For Part 22, I added an export function to Crafty, so I could select chunks of landscape and write them out. This is how I added the buildings to the green asteroid and other demos. Crafty was only reading and writing its own format however.

This week, I've written a new program McrView which reads the Minecraft files directly and displays them with my rendering code. Note that since I don't have the algorithm Notch uses to generate landscape, there will be holes in the world wherever a chunk has never been saved by Minecraft.

You'll only want to run this if you already have Minecraft save files on your machine. They are in a different place in each OS. If you run Minecraft, each world is listed on the "Select World" loading screen. Take the name of the world you want and set the "world" option in the options.xml file. It defaults to "New World", which is the default Minecraft world name. I think older versions of the program may use "World1".

If you have the save files from a server, first install them as a world for local use. Do that by copying the world into the Minecraft "saves" directory. Run Minecraft and enter the world you've copied. It will convert all the files (in Alpha format) to the newer Beta format my viewer reads. On the various operating systems, find Minecraft files here:

  • Windows: C:\Users\UserName\AppData\Roaming\.minecraft\saves
  • Linux: ~/.minecraft/saves
  • Mac: ~/Library/Application Support/minecraft/saves

Once you have options.xml pointing at a world, just run McrView. There's no actual export function yet, since I want to get Crafty and SeaOfMemes running again first. But you can look at your buildings with my code. As in all the Minecraft data I've imported, there are many block types I don't support yet.

The UI is the usual:

  • WASD for movement. Arrow keys also turn and move.
  • Page up/Page down to change your height. Blank and X also work.
  • + and - to change your speed.

I've set the view distance to 160, which seems close to what Minecraft uses. To increase it, edit the "viewDistance" attribute in options.xml. You may also want to increase "displayMemory" and "systemMemory" to reflect your hardware. The default is 200meg for the display, which is pretty conservative.

For Windows, download McrView Part 32 - Windows.

For Linux, download McrView Part 32 - Linux.

The Mac version has a bug which I haven't found. I'll update this part when I do.

Let me know if it all works for you.

How Embarrassing

I started Part 1 with the idea of using an Octree to compress the array of cubes that make up the world. It's a fine data structure for that. It is quick to traverse, so I can build the vertex and index lists, and quick enough when adding and removing bricks to handle interactive use. But "quick" is a relative term, and I've run into problems with the Octree performance twice now.

The first time was in Part 21, when I was trying to do lighting. I need to cast rays from the vertexes to the light sources. Doing that means stepping through all the brick positions along the ray. In the Octree, that means that at each step, you start at the top and work down through up to 7 levels of the tree, comparing your requested coordinate against that level. This takes time. When you are stepping lots of rays through lots of cells, it takes a lot of time.

I pulled out the core piece of code and timed it. I thought about what I could change to speed it up, and I didn't come up with anything. So I put lighting aside. I thought perhaps I could unpack the tree into a big array and do all the lighting calculations there. I didn't get a timing on how long it would take to traverse the tree and build a big array (32K cells), but I knew it wouldn't long.

The second problem with performance came in Part 30. I wanted to build landscape procedurally, then turn it into cubes when you got close. This means filling a chunk with bricks based on the height of the landscape. It was taking my machine about 10 milliseconds to do that. That's not bad if you are only doing one, but when I keep the entire nearby landscape as cubes, there are many new chunks to do as the user moves. It just wasn't quite interactive. I don't want to have problems like that now, before I have any game logic going on in the background.

There were other problems. The Octree compresses the data when groups of bricks are identical. A 2 by 2 by 2 group of stone bricks turns into a single size=2 node in the tree. However, each piece of data I add to a brick makes them more unique and less compressible. For example, I added a visibility flag, showing which faces are exposed. Now a 2 by 2 by 2 brick won't compress unless it's completely buried. If a face is showing, the visible face has different data in it than the buried face, even if it's all stone.

So with great regret and embarrassment, I have ditched the Octree data structure. Right now, I'm not compressing the brick array at all. When I finish with vertex lighting and implement the rest of the Minecraft brick types, I'll do some simple compression. Probably a run-length encoding of the bytes will do the trick.

The simple byte array format actually lets me get rid of the visibility flags. I had put them in because it was too slow to check all six neighbors of a cube whenever I needed to do something with it. With the data all in an array, it's almost as quick to check the neighbor cells as to check flags. So even though I'm not compressing, I'm storing half as much data. The demo runs fine without any compression at all.

Site Redesign

I'm still fussing over the look of the home page, but I have changed the individual pages to the new style. The main difference, other than the color scheme and font on the top and bottom navigation bars, is the width. I've set a "max-width: 100ex" style element, which should keep the width reasonable. This replaces the "70%" rule I was using.

For narrow windows, this should be an improvement, since it won't waste any space to the right. For people with very wide windows, you'll get a more normal column width, instead of 70% of your screen width. Let me know if you have any trouble with this.

I'll put in the new home page when there are good demos of both Crafty and SeaOfMemes. I might also play with the colors and background image a bit more.

Home       About Me       Contact       Downloads       Part 31    Part 33   

blog comments powered by Disqus