Note: After doing some more research, and figuring out some more details about how RT3 works, I no longer recommend this procedure.
The short version is that heightmaps should not be resized after being exported from MicroDEM, unless the "resizing" is limited to cropping off a few pixels to get it to 64+1. Stretching or squashing the heightmap, even slightly, will create artifacts in the game map.
I've done a few more experiments, and have figured out something useful.
My usual procedure is to export the DEM as .BMP, then resize it in Photoshop to get the proportions matching for vertical and horizontal scale (DEM's are always a bit out for scales on those two axes). Doing this gets the final .TGA in scale overall, so the map is a free from stretching or squashing as it is possible to get it. I know a lot of people don't bother with this resizing, and just use maps however they get spat out, but I like doing it.
Anyway, this causes problems around coastlines. Resizing the image results in blurred pixels where the pure red of the ocean joins the blue/green of the elevation pixels, giving a fringe of pixels that don't have valid values for the height scale. Getting around that is easy enough. I used to frig around with more complex ways of doing it, but the simplest way is to use the Nearest Neighbour algorithm.
The Nearest Neighbour algorithm always gives pixels that are either pure red or completely transparent, meaning the new layer will make a clean ocean regardless of scaling. This part is fine, but the catch is that
Nearest Neighbour is total crap for resizing the blue/green elevation pixels. This is where I've had to get cunning.
To get the best results,
duplicate the .BMP before resizing it. This way you can use two different resize algorithms on the two separate images: Nearest Neighbour on the image you'll take the red ocean from, and the default Bicubic on the image you'll take the blue/green elevation pixels from.
Why? Well, if you use Nearest Neighbour on the whole thing in one go it will import just fine, but if you are making a map with fairly gentle slopes the result without smoothing will be a series of terraces, rather like shallow mesas stacked on top of one another. It took me a while to realise the cause, and I'm pretty sure it's down to error margins on the DEM's themselves.
The table of colours we use with MicroDEM is theoretically capable of a 1 metre elevation resolution, but DEM's are only accurate to 10 or 15 metres elevation. This means if all the raw data gets shoehorned into blocks of 10 or 15 metres, then that shoehorning into those blocks would create the stacked terraces when imported into RT3. It has to, if the raw data is rationalised so that the finished DEM has all its pixels within the error bars. This seems to be what is happening. I found this out when I figured out Nearest Neightbour was the solution to ocean edges, and tried it on the whole thing. Terraces everywhere, if you don't use the default smoothing of 1 and you set it to 0.
Ok, so how was I getting good results on the trial Orient Express maps without smoothing? That was just before I figured out the Nearest Neighbour trick, so all the elevation pixels on those maps were resized with Bicubic. Having thought about that too, it became clear that this would also blur the elevation pixels at boundaries between two colours but, and this is the tricky bit,
in the case of elevations it actually appears to be an advantage.
What it must be doing is effectively injecting intermediate values of the "bluescale" in between the values that would ordinarily result in the terraced effect. In other words, it's automatically applying a low level smoothing by interpolating extra elevation values, but without going the full melted blob that the RT3 default smoothing gives. It is basically a way of getting a fractional smoothing value that RT3 ordinarily doesn't allow on import, and the amount of it will be proportional to the amount of blurring caused by the resizing the .BMP.
There's another minor catch here, of course, in that sometimes the intermediate values it injects are not quite valid for that pixel, meaning it produces noticeable artifacts when imported. These usually appear as very small and very distinct needle-like peaks on an otherwise smooth surface. These are not very common, and are easy to spot and easy to knock back when spotted. Even allowing for such artifacts, I still think the overall result is better for maps where you want accuracy on complex terrain.
The upshot of all this is that I've nailed down my procedure like this:
1/ Select the cropping limits for the map, based on whatever latitudes and longitudes will give an equal vertical and horizontal scale once the image is forced to the desired multiples of 64. This requires a bit of calculation, but nothing particularly difficult.I use a calculator and this tool to figure it out:
http://www.csgnetwork.com/degreelenllavcalc.html
2/ Before importing the DEM into MicroDEM set the default map size (Options > Maps) to whatever you want for the larger dimension of finished TGA heightmap. The reason for doing this is it means MicroDEM will automatically export the temporary .BMP with it's larger dimension correct for the finished TGA. Note that this will
only sort out the larger dimension. You cannot use MicroDEM's default map size to force a different aspect ratio. That still has to be done by resizing in Photoshop.
3/ Now import the single DEM or multiple DEM's (which will automatically be merged). Then right click on the image > Modify map area > Keyboard corners > type in whatever values give you your correct cropping limits (one 5 degree DEM equals 6000 units in the "Keyboard corners" inputs). Hit enter. Hey presto, map cropped to the right limits.
4/ Make sure you are using the correct colour scale (right click on the image > Elevation colours > Colours from table > ELEV_COLORS_0-9999.dbf).
5/ File > Save image. You now have a BMP with one dimension correct.
6/ Open Photoshop or whatever beastie you use. Open the BMP. Duplicate it now.
8/ Resize the incorrect dimension to make it fit the 64+1 thang. Resize both images, but use the two different resize algorithms: Nearest Neighbour on one, and Bicubic on the other.
9/ For added insurance, on the image that was resized Bicubic you can ditch the red channel (Image > Adjustments > Channel mixer > Red to -200). This will make sure that any traces of red that may have bled past the ocean will be removed from any visible pixels, leaving just the desired blue and green channels.
10/ Select the red from the Nearest Neighbour image (tolerance set to 0) and copy paste that selection onto the other (Bicubic-resized) image.
11/ Do a final check around coasts for tiny islands and indentations that are too small to be useful. Either paint them out with pure red on the red layer (if trying to get rid of a small island) or crop them out of the red layer (if wanting to get rid of a tiny patch of useless ocean inside the shoreline).
12/ Now you can save the thing as 24 bit TGA and import it into RT3. Instant game map. Mess around with height modifers until you get something you like.
This is all a lot easier to do than it is to read about, once you've done it once or twice. It's actually a pretty simple procedure and goes quickly.
