GuideGridTool - Converting between guide grids and images

Talk about your new mod or map here

Moderators: bert_the_turtle, jelco

User avatar
jelco
level5
level5
Posts: 6018
Joined: Sat Feb 18, 2006 7:45 am
Location: Cygnus X-1
Contact:

GuideGridTool - Converting between guide grids and images

Postby jelco » Sun Jul 05, 2009 11:19 pm

GuideGridTool
v0.2.2 (August 15th 2009)

Downloads
  • Latest public version, Python source + Win32 executable: 0.2.2

Alright, so because I offered to do this on a whim and then was slightly pressurized into actually doing it by TGF :P here's a small Python script that can convert guide grids to image files and vice versa.

First things first: this tool was created for use during the Multiwinia beta so it's been in development for a short while now (let's not exaggerate, 2 weeks) and I figured it might be useful to Darwinia modders as well - hence this post is basically a copy-and-paste job from the original topic. My main working version includes some minor functionality only applicable to the beta so I've had to remove certain parts from this post, hence it might come across as slightly sloppy or incomplete. If something is unclear, don't hesitate to ask.

There's a Win32 executable available so that you can run it out of the box but hardcore users (or non-Windows users) might want to go all the way. To run this script you'll first need to have Python installed (2.6, not 3.0), after which you need to get the Python Image Library. After that you can simply invoke ggt.py on the command line and you're set.

Syntax:
ggt.py/ggt.exe --grid-to-image grid.txt image.jpg
ggt.py/ggt.exe --image-to-grid image.jpg grid.txt
ggt.py/ggt.exe --image-to-map image.jpg map.txt
ggt.py/ggt.exe --map-dump map.txt

All commands have shortened versions (-g2i, -i2g, -i2m and -md respectively) and you can find some help when running the app without arguments. Needless to say, it's a command-line app.

Currently the grid file has very strict requirements because I didn't feel like coding in flexibility at this hour (yay almost 4AM and a diploma ceremony 'tomorrow' night). It must be exactly the guide grid you want - it can't even have a newline at the end.

There's two example files you can download. These files are each other's counterparts so converting between them should be an easy tryout. It's a center guide grid from one of my own Multiwinia maps, one with steady slopes to four sides.
Example grid
Example image

Colours in images, very simple: White is high, black is low. If you convert a colour image to a guide grid it will take the average value of red, blue and green for each pixel to calculate the value it wants - this means that a 100% red, blue or green pixel is equal to a 33% black pixel and 100% cyan, magenta or yellow is equal to 66% black. This might come across as confusing, and if it is, just don't try it. :P

As for image files it'll understand, that depends entirely on the PIL. It supports PNG, JPEG and BMP as far as I'm aware and since those are the most common formats I guess that'll do. I don't know about image sizes that create unworkable guide grids for MW, but the highest res guide grids equal a 31x31 image, so stick to that if you want to play it safe. If you want to experiment, do try it and do tell me the results - note that it must always be a square however. For some reason PNGs don't seem to work out perfectly - full white works out as roughly 50% in the resulting grids - so sticking to BMPs and JPEGs is the best way to go. When using JPEGs, make sure to keep the quality level at 100, because otherwise you'll have compression artefacts messing up your grids.

The core feature of the app is --image-to-map/-i2m, which can split up an image into multiple smaller images to fit with guide grid resolutions (you are restricted to certain resolutions nonetheless, see README). When using this it will ask you for a size ratio (controls the size of the landscape tiles, experiment to find out what works best for you - I prefer something between 20 and 30), a maxheight value, and offsets for x, z and y (note that due to Chris' weird naming convention, y is the vertical dimension and x and z horizontal, hence they're asked in that order). If the output file you supplied is a pre-existing txt file it will try to handle it as a map file and ask you if you want to overwrite the file or append the grids to the map. Note that landscape tiles are not truncated when appending to a map so if you want to try multiple times to e.g. experiment with the ratio, use backup files.

The overlap between multiple tiles when using -i2m is not very clean yet, but for the moment it does the job. I'm going to have to look into a slightly more professional solution later on (currently the tiles just overlap 10%).

Even when using an image that is just for one tile and it doesn't need to be cut up into multiple tiles, use -i2m because it includes formatting and saves you a lot of hassle with the offsets and height/size definitions. The other two commands are generally 'raw' commands, although formatting with a default set of values is possible for the -i2g feature ("ggt -i2g image.jpg grid.txt -f" gives you offsets of 0 and a maxheight of 200). On top of this, -i2m is the only command that currently does and will ever support writing into pre-existing map files.

There is a lazy mode for the image-to-map feature which will save you a lot of hassle. It is mainly intended for testing purposes. It will enter default ratio, height and offset values, will automatically try to append if the file pre-exists and falls back to overwriting if this fails. The default ratio is 20, the default maxheight is 200, and the default offsets are 0.

The map-dump feature will scan the supplied map file for any landscape tiles with guide grid definitions and will save them in separate JPEG files named after the original map filename and numbered in the order in which they appear in the map file. It doesn't include any offset, height or size values, just the guide grids.

Jelco
Last edited by jelco on Sun Aug 16, 2009 2:13 am, edited 3 times in total.
User avatar
The GoldFish
level5
level5
Posts: 3961
Joined: Fri Mar 01, 2002 9:01 pm
Location: Bowl / South UK
Contact:

Re: GuideGridTool - Converting between guide grids and image

Postby The GoldFish » Mon Jul 06, 2009 5:09 am

jelco wrote:Will there ever be a map-to-image feature? Honestly I doubt it given the total lack of use of such a feature but if I get bored enough I may just try to write it. Pretty complicated though when you get multiple overlapping tiles with different maxheights and everything...

Jelco


Oh man that sounds so hard ;-;

Seriously though, if you don't know off hand which grid guide you want, it would be nice for it to just simply be able to iterate through a map and produce an image for each grid guide, and that's all, not do anything special, and just number the images as they come out of the file; tells you everything you need to know. Maybe not call it map to image, but, just dumping all the guides out could be helpful rather than doing each one by hand.

As an aside, I've been using pngs exclusively while using your app, which I have been helping you test, and have not noticed any problems at all, the only barrier to progress was that I have apparently been using interlaced pngs in general for a very long time, which PIL doesn't support at all.

As a further aside, I would like to endorse this handy little app. I'm probably never going to use "-image to map", the supposed core feature of this, because I can't imagine memorising the offsets and scales of how I want to inject it into the map, I just want it IN the map, to the end that I'll probably make a grid guide in the level and paste a grid string in myself, rather than use this feature. Thinking about it, it would be nice to be able to inject it into a map with some default values, so you can just use the map editor to sort it out later without having to first create a shell guide. The objective there is to just get it into the map so you can edit it yourself.

But anyway the reason I endorse it is that editing grid guides, especially to be complicated, such as long chained ramps, can take hours to get perfect, and are easy to ruin by accident. You can use this app to very easily do a lot of things that you simply would spend an eternity doing otherwise.
nihilesthetics2
level2
level2
Posts: 101
Joined: Mon Dec 12, 2005 11:18 pm

Postby nihilesthetics2 » Mon Jul 06, 2009 6:00 pm

I love it when people post up cool little utilities like this. Well done.

Anyway, I remember the largest map size was 127*127.

Edit: I dont own Multiwinia, but I would love to know if it could go higher than 127*127 (I know, I know I'm sad that way). Anybody want to test and post up the results ? The reason this was the limit in Darwinia was because of the maximum file size of 65K, but they might have changed this.
User avatar
NeatNit
level5
level5
Posts: 2929
Joined: Mon Jan 28, 2008 2:41 pm
Location: Israel
Contact:

Postby NeatNit » Mon Jul 06, 2009 7:32 pm

Multiwinia's level editor isn't released yet.


As for the real max resolution, I have no idea, but I prefer to believe the OP saying it's 31x31. No reason, just that he's way more active on the forum and I believe him more :P
User avatar
jelco
level5
level5
Posts: 6018
Joined: Sat Feb 18, 2006 7:45 am
Location: Cygnus X-1
Contact:

Postby jelco » Mon Jul 06, 2009 7:40 pm

Yes, there are two higher resolutions than the max the editor uses (31x31 is res 5, beyond that you have res 6 of 63x63 and res 7 of 127x127). I've found however that maps load very slowly in Xwinia when you use resolutions higher than 5, so it's something I discourage. Any resolution above 7 requires so many characters on one line (res 8 is 255x255 so that's 65025 pixels which equals 130050 characters) that Multiwinia crashes (not sure about Darwinia's reason, but I'm guessing it might be the same). The error Multiwinia gives is "Text file contains line with more than 65536 characters" which basically seems to suggest that any resolution above 7 is ruled out.

My advice though: stick with resolutions 5 or lower. Higher resolutions will rarely be useful, it's easier to make composite landforms with multiple tiles to achieve the same effect.

I'm wrapping up a new build at the moment, stay tuned!

Jelco
User avatar
jelco
level5
level5
Posts: 6018
Joined: Sat Feb 18, 2006 7:45 am
Location: Cygnus X-1
Contact:

Postby jelco » Mon Jul 06, 2009 8:05 pm

0.2.0 is up. Two important new changes:

  • The lazy mode (switch -l at the end of the command) saves you the hassle of having to provide data. It will enter default ratio, height and offset values, will automatically try to append if the file pre-exists and falls back to overwriting if this fails. It can save you a lot of time but is generally not suited for anything but testing purposes.

    Example syntax:
    ggt -i2m image.jpg map.txt -l
  • The map-dump feature will scan the supplied map file for landscape tiles with guide grids and save all of them to numerically ordered images named after the original map filename.

    Example syntax:
    ggt -md map.txt

Jelco
User avatar
NeatNit
level5
level5
Posts: 2929
Joined: Mon Jan 28, 2008 2:41 pm
Location: Israel
Contact:

Postby NeatNit » Mon Jul 06, 2009 8:09 pm

jelco wrote:
  • blabla
  • blabla
what's with the extra [*] in the middle?
User avatar
Phelanpt
level5
level5
Posts: 1837
Joined: Thu Aug 10, 2006 4:20 am
Location: Portugal

Postby Phelanpt » Mon Jul 06, 2009 11:08 pm

Oh, that one is the best, you can use it to [NDA] your [NDA] as much as you want! :P
nihilesthetics2
level2
level2
Posts: 101
Joined: Mon Dec 12, 2005 11:18 pm

Postby nihilesthetics2 » Tue Jul 07, 2009 4:24 pm

jelco wrote:My advice though: stick with resolutions 5 or lower.


Agreed - the lower the requirements for your mod, the more people will potentially play it. Sometimes, however, you just want to push a thing to see how far it will go before it breaks :)

jelco wrote:"Text file contains line with more than 65536 characters"


Interesting - it says the line is bigger than 65K, not the file like I thought it was with Darwinia. Perhaps multiwinia is different after all. If it is a line limit you could have several tiles of 127*127. I wonder how many 127*127 tiles you could get on there before ... oh, there I go again.
User avatar
briceman2
level2
level2
Posts: 123
Joined: Wed Dec 12, 2007 4:30 am

Postby briceman2 » Thu Jul 09, 2009 2:50 am

nihilesthetics2 wrote:Interesting - it says the line is bigger than 65K, not the file like I thought it was with Darwinia. Perhaps multiwinia is different after all. If it is a line limit you could have several tiles of 127*127. I wonder how many 127*127 tiles you could get on there before ... oh, there I go again.


Line length limits are courtesy of the OS. Even Linux imposes limits based on line buffer sizes which are hardcoded at kernel compile time (IIRC).

The max .map or .mission file size = unlimited. Just recall that you can save game with tens of thousands of units on fast fast machines. I have used a perl script to map images into grid guides in the past. It's easy to use 16 or 64 level 7 grids in a single map file. Although I agree that using a lot of well-placed level 5 grids is a better solution than a few level 7 grids. ***BUT*** there is a hitch (as always)... it is often very difficult to get gridded tiles to abutt each other without some kind of artifacts. So if you want to create a smooth mega-terrain it is sometimes better to go with one level 7 grid instead of 16 level 5 grid tiles in a "grid". The edge artifacts are very sensitive to the absolute coords of the grid tiles IIRC. You can use the map's cell size (can't remember the real term) to calculate the "natural" boundaries of tiles -- but can't recall the exact formula right now.

Under the linux version of Darwinia I've never experienced any slowdowns when using very complex map files, including multiple level 7 grids. This may be a Windows OS thing when handling ultra-long text lines??

-brice
nihilesthetics2
level2
level2
Posts: 101
Joined: Mon Dec 12, 2005 11:18 pm

Postby nihilesthetics2 » Fri Jul 10, 2009 2:29 pm

briceman2 wrote:Line length limits are courtesy of the OS. Even Linux imposes limits based on line buffer sizes which are hardcoded at kernel compile time (IIRC).


I dont get it. A file is just a big lump of bytes, some of which just happen to be newline or carriage return characters. I cant see how the OS could limit our ability to treat any number of bytes as something that we can arbitrarily define as a 'line'. I could see the compiler of even the libraries/code itself being the limit here, but not the OS. But then Im not really a programmer - so you probably outrank me on this one :P

briceman2 wrote:Under the linux version of Darwinia I've never experienced any slowdowns when using very complex map files, including multiple level 7 grids


I got (noticeable) lag with a single level 7 grid on Windows - Mark one up to Linux :)
User avatar
jelco
level5
level5
Posts: 6018
Joined: Sat Feb 18, 2006 7:45 am
Location: Cygnus X-1
Contact:

Postby jelco » Fri Jul 10, 2009 2:33 pm

briceman2 wrote:Line length limits are courtesy of the OS. Even Linux imposes limits based on line buffer sizes which are hardcoded at kernel compile time (IIRC).

I have been told by um...reliable sources that it doesn't have to do with the OS. If you have lines that long it's still possible to open the files in apps like Notepad++ (although they are incredibly slow and prone to crashing if you do that) so it doesn't look like an OS limit anyway. Why exactly MW imposes this limit I don't know, but it probably has a good reason.

As for the overlap of multiple tiles (using the image-to-map feature): it simply lets them overlap by 10%. This solves most gaps and looks pretty good in 9 out of 10 cases - sometimes though when you have drastic changes on such a seam it can look weird and that would require different solutions indeed.

Jelco
"The ships hung in the sky much the same way that bricks don't."
- Douglas Adams
User avatar
jelco
level5
level5
Posts: 6018
Joined: Sat Feb 18, 2006 7:45 am
Location: Cygnus X-1
Contact:

Postby jelco » Sun Aug 16, 2009 1:16 am

0.2.1 is up.

  • Most importantly, the overlap for image-to-map has been shinified. It is no longer a hack which produces bad-looking results, but it actually accounts for the overlap by making sure the pixels at the edges of meeting tiles match. As a result, the resolutions for input images now adhere to stricter standards. See the README for details.
  • A minor bug has been fixed which made the application crash when entering an invalid value for the ratio

Jelco
Last edited by jelco on Sun Aug 16, 2009 12:49 pm, edited 1 time in total.
"The ships hung in the sky much the same way that bricks don't."

- Douglas Adams
User avatar
jelco
level5
level5
Posts: 6018
Joined: Sat Feb 18, 2006 7:45 am
Location: Cygnus X-1
Contact:

Postby jelco » Sun Aug 16, 2009 2:18 am

Immediately followed by 0.2.2.

  • The image-to-map feature now also compensates for the odd behaviour Xwinia exhibits when the maximum pixel height in a guide grid is not the maximum possible value. It corrects tile heights and leaves out tiles entirely if they turn out to be empty (something which was neglected previously and could create weird results if certain areas of an image were completely black). This is just an under-the-hood change and doesn't require you to do anything differently when using the tool, but it will reduce (hopefully eradicate completely) unexpected results to do with this Xwinia behaviour.

Jelco
"The ships hung in the sky much the same way that bricks don't."

- Douglas Adams
User avatar
NeatNit
level5
level5
Posts: 2929
Joined: Mon Jan 28, 2008 2:41 pm
Location: Israel
Contact:

Postby NeatNit » Sun Aug 16, 2009 12:36 pm

jelco wrote:0.2.1(r) is up.
(r)?

Return to “Mod Projects”

Who is online

Users browsing this forum: No registered users and 2 guests