I oftenly get asked how to load and create SCMAP files. To answer those questions i made this bunch of VB inspired pseudo-code that demonstrates loading of an SCMAP from a stream.
(ReadString without a given length always reads a null-terminated string.)
Count = ReadInt32 For i = 1 To Count LoadWaveGenerator Next
ReadString ' always "No Tileset"
Count = ReadInt32 ' usually 6 For i = 1 To Count LoadLayer Next
Unknown7 = ReadInt32 Unknown8 = ReadInt32
Count = ReadInt32 For i = 1 To Count LoadDecal Next
Count = ReadInt32 For i = 1 To Count LoadDecalGroup Next
ReadInt32 'Width again ReadInt32 'Height again
Dim Length As Int32 Count = ReadInt32 ' should always be 1 For i = 1 To Count Length = ReadInt32 If (i = 1) Then NormalmapData = ReadBytes(Length) Else CurrentOffset += Length End If Next
Length = (Width / 2) * (Height / 2) WaterFoamMap = ReadBytes(Length) 'Obviously not used.. each byte is 00 WaterFlatnessMap = ReadBytes(Length) 'Obviously not used.. each byte is FF WaterDepthBiasMap = ReadBytes(Length) 'Obviously not used.. each byte is 7F
TerrainTypeData = ReadBytes(Width * Height)
If (FileVersionMinor <= 52) Then ReadInt16 'always 0
Count = ReadInt32 For i = 1 To Count LoadProp Next
end fucntion
This post has been edited by HazardX: Feb 21 2007, 21:41 PM
In notepad, characters 1-3 (corresponding w/ bytes 00-02) always spell "map" Byte 10 float and Byte 04 in the heightmap apparently need to correspond with the "mapsize" in the senario file.
open up any of the scmap's with notepad. do ctl+a. change font size to 1. open file up again (should still be size 1), scroll down slowly. See something familiar? (for some reason the rightmost 1/4th is displaced to the far left of each "image" (if you want to reverse the size change just do ctl+a and change size to 10)
OK! I figured out that there are several sections:
1 header 2 heightmap 3 gibberish 4 texture files and (placement?) 5 blank space/divider 6 lots of Y's 7 lots of [] <- rectangles 8 texture map 9 prop data (trees and rocks)
Maps are contained in subdirectories of the Supreme Commander\maps directory. A map subdirectory has a name of the format SCMP_###, where # is a three-digit positive integer. The subdirectory contains a SCMAP file and several LUA scripts, each prefixed with the map's directory name (not case sensitive). While all the maps working with the beta have only three scripts (save, scenario, and script), a "hidden" PerfTest map has quite a few more. With names like "aeon1plan" and "uefplan," it sounds like the AI will be powered by LUA scripts distributed with the map.
The LUA scripts distributed with the maps have the following format:
save.lua -- Contains a Scenario object. This object defines Props, Areas, Markers (in MasterChain and Chain objects), Orders, Platoons, and Armies.
scenario.lua -- Contains Scenario metadata, including the map's name and description, as well as directions to the SCMAP, save.lua, and script.lua files (as relative paths using the Supreme Commander folder as root). Seems that only the scenario.lua script must be named properly, and it can point to anywhere under the SupCom folder. The map size is also given, as grid dimensions.
script.lua -- Contains event handlers for the Scenario. Known event handlers are: OnPopulate, OnStart.
I'll be taking a look at the SCMAP files over this week, we'll see how that goes. With any luck, we should have a simplistic editor in less than two weeks.
(Oh, and varanus-sapiens -- You might want to get a hex editor for this, Notepad doesn't help much when it comes to binary formats.)
This post has been edited by Adam Blinkinsop: Dec 25 2006, 19:08 PM
well, yeah, I know I need a hex editor... I've needed one for years... Preferably, I'd use both a hex and notepad since there are things that do not show up in one but do in a different. Can you visually edit the texture or hightmap when looking only at a hex editor? mmm, not really, it shows up much more clearly on notepad at size 1 font (allows you to see a "larger picture" ), but in notepad, much of the other stuff appears as gibberish.
With any luck, we should have a simplistic editor in less than two weeks.
Yes, i'm already working on an editor. Editing all markers is possible already. Editing units can easily be added within minutes. Modifying the heightmaps will be ready in a few days. Unfortunately i'm not at home at the moment. i won't be able to continue working on it earlier than tomorrow evening. However, modifying the Heightmap will only be possible by replacing those values in the existing files, since we aren't able to create a whole new SCMAP file for now. I hope you are better than me in decrypting this annoying TTerrain section (just behind the heightmap data).
BTW: i used my editor to place all markers in my Tropical Crusade map, since no data of that kind was shipped with the map (which is hidden in the perftest folder).
QUOTE(varanus-sapiens @ Dec 26 2006, 12:12 AM)
Can you visually edit the texture or hightmap when looking only at a hex editor?
You can't "edit" it with notepad neither, since it will completely mess up everything on saving. It is just a lucky coincidence that you are able to see the map structure when viewed in notepad, but that won't help us much. We need to decode the format bytewise to be able to modify or create those files and the only way to do so is a hex editor. You are right, a hex editor is far from being "visual", but it's the only way.
This post has been edited by HazardX: Dec 26 2006, 02:14 AM
I believe the Texture Map starts at byte 001C565659, offest 01857113, I forgot where it finishes thiough since I lost my notes >.>"
Yes, those values look like they are texture indexes for each space on the map.
BTW: We should define one map as the default map for our testing since the offsets differ for each map because of things like map size or texture names. i'd suggest SCMP_015 (Fields of Isis) for this.
[EDIT] I've attached a screenshot of Isis with a modified heightmap. Just replacing those heightmap values works great.
This post has been edited by HazardX: Dec 27 2006, 16:28 PM
Well, I have to use what I have. I'm on vacation too so I can't really accomplish all that much, and I have some modding for another game, and bleh... Any way this thread is mostly for results and harvesting info, and wordpad does the latter effectlivly for the visuals. @HazardX That done by your program or what? pretty nice... What about the size #'s at byte's 04 (map) and 10 (header) for an 81x81 map? think isis canbe "interpolated" or merely stretched to achieve an oversized map.
Well, I have to use what I have. I'm on vacation too so I can't really accomplish all that much, and I have some modding for another game, and bleh... Any way this thread is mostly for results and harvesting info, and wordpad does the latter effectlivly for the visuals. @HazardX That done by your program or what? pretty nice... What about the size #'s at byte's 04 (map) and 10 (header) for an 81x81 map? think isis canbe "interpolated" or merely stretched to achieve an oversized map.
Yes, thats right. We need any information and every hint we can get.
Modifying the heightmap is relatively easy. Open a map with a hex editor, copy all heightmap data to a new file with a RAW suffix, open it with Photoshop, select 16bit Grayscale, paint everything as you want it to be (brighter = higher), save it, then open the file again with a hex editor and copy all data back over the heightmap data. Currently i have Export/Import Heightmap functions in my editor to avoid those annoying copy steps, but i'm working on some tools like raise, lower and flatten that will be available directly in the editor. BTW: The heightmap loaded in Photoshop will be quite dark, since the terrain is very low by default.
It would be possible to strech Isis to a bigger map size, but not yet. We have to decrypt all major parts of the SCMAP format first, since the amount of data grows with the mapsize and we need to know where we have to insert the data.
This post has been edited by HazardX: Dec 27 2006, 18:56 PM
Go the simplistic way of modifying the hightmap, I was just editing hex values -.-" (lol) Nice work once again HazardX
On another note:
QUOTE
Byte 03: byte - Length of the header (used to jump directly to the preview image)
I'm still rather new to this all but I'm wondering how you use this byte to jump directly to the preview image? Do you use that length to go from the start of the header to the end and then following the header is the preview image?
This post has been edited by Arthain: Dec 29 2006, 00:33 AM
I'm still rather new to this all but I'm wondering how you use this byte to jump directly to the preview image? Do you use that length to go from the start of the header to the end and then following the header is the preview image?
Yes, the byte just contains the length of the header data in bytes. when you read this byte, just skip as many bytes and you'll be directly at the location where the length of the preview image data (and afterwars the data itself) is stored. it is just meant to speed up loading when you don't care for anything else. in most filetypes those length values are stored for each section so that you are able to reach every section of the file without loading a lot of unwanted clutter. however, GPG didn't do this for the other parts of the file, which makes it hard to determine where a section starts and ends.
Yeah. It can be really annoying for me at least where the preview data ends and the heightmap begins. Well trial and error should get me there... eventually. Are all the preview images a set size because then we could tell how big the section is by calculating the number of pixels in the image which isn't very hard to do... I might go try it.
--------------------- EDIT ------------------- Interesting find: There must be a setting for Water height because if you change a Heightmap on a land map such as Fields of Isis to incorperate what would be water you just get a huge canyon...
This post has been edited by Arthain: Dec 31 2006, 02:44 AM
Firstly I know this is a double post I'm doing it so that people know theres a new post and it'll bump the topic up to the top.
This is regarding the Terrain Overlay. Ok, now I'm gonna try to work on it during the next week if I get some time but I had some questions mainly directed at HazardX since he knows the most xD Firstly I think that the terrain overlay will be made up of a tile set so we won't be able to view it using a similar method as that used as viewing the heightmap, correct? Secondly I'm going to look through the linking scenario and texture files to see if I can find either a link between them or a group of tilesets, given that I can find it we should be able to link tiles to their hex codes.
Just wondering what your thoughts on that are and if they are wrong or if theres anything you can suggest. ^.^
well, this thread isn't up-to-date anymore. i've found ALOT of data structures in the last days and am able now to read the whole SCMAP filestructure and build a new one out of it with changed values. i haven't updated this thread since i'm still busy verifying all this stuff. the part that looked like a texture index list is just a kind of "passability map" only indicating how easy units can move on this field. regardings texturing there is an included DDS image that seems to be a color-coded texturemap. each base color R, G and B represents a texture. those can be drawn and mixed easily that way. i wasn't able to retexture a map yet, though. something seems to be still wrong.
Awesome! Thanks very much. I'm going to have a lot of fun with this. There's just one part that's unclear to me at the moment, and that's the texturemap. I know your editor imports it as a 4 channel (RGBA) DDS file, but I'm not sure where that is in the format, as described by your pseudocode, unless it's the: TexturemapData = ReadBytes(Length) If that is it, could you elaborate at all on the format of TexturemapData? (I'm really hoping you don't just say "it's DDS format", because I have no idea how to read that format.)
Awesome! Thanks very much. I'm going to have a lot of fun with this. There's just one part that's unclear to me at the moment, and that's the texturemap. I know your editor imports it as a 4 channel (RGBA) DDS file, but I'm not sure where that is in the format, as described by your pseudocode, unless it's the: TexturemapData = ReadBytes(Length) If that is it, could you elaborate at all on the format of TexturemapData? (I'm really hoping you don't just say "it's DDS format", because I have no idea how to read that format.)
I am really sorry, but it IS in DDS format. All textures used by SC are. I'm using DirectX to load and save the textures, but i don't know if there are any DirectX or DDS libraries for java.
I am really sorry, but it IS in DDS format. All textures used by SC are. I'm using DirectX to load and save the textures, but i don't know if there are any DirectX or DDS libraries for java.
Do you know if the DDS is stored compressed, in the map file? I imagine it probably is. I've found some open-source C DDS code, but it leverages the card's OpenGL to do the texture compression and decompression, which I'm not sure I'll be able to do in Java. If I can port the rest of the library though, at least I'll be able to read and write uncompressed DDS files.
Do you know if the DDS is stored compressed, in the map file? I imagine it probably is. I've found some open-source C DDS code, but it leverages the card's OpenGL to do the texture compression and decompression, which I'm not sure I'll be able to do in Java. If I can port the rest of the library though, at least I'll be able to read and write uncompressed DDS files.
I'm not sure since DirectX does all the work. I think that the A8R8G8B8 format is saved uncompressed, but i can't guarantee it.
(ReadString without a given length always reads a null-terminated string.)
function LoadWaveTexture
ScaleX = ReadFloat
ScaleY = ReadFloat
TexturePath = ReadString
end function
function LoadWaterShaderProperties
Unknown1 = ReadArray(Of float)(20)
TexPathWaterCubemap = ReadString
TexPathWaterRamp = ReadString
Unknown2 = ReadArray(Of float)(4)
For i = 0 To 3
LoadWaveTexture
Next
end function
function LoadWaveGenerator
TextureName = ReadString
RampName = ReadString
Position = ReadVector3
Rotation = ReadFloat
Velocity = ReadVector3
LifetimeFirst = ReadFloat
LifetimeSecond = ReadFloat
PeriodFirst = ReadFloat
PeriodSecond = ReadFloat
ScaleFirst = ReadFloat
ScaleSecond = ReadFloat
FrameCount = ReadFloat
FrameRateFirst = ReadFloat
FrameRateSecond = ReadFloat
StripCount = ReadFloat
end function
function LoadLayer
PathTexture = ReadString
PathNormalmap = ReadString
ScaleTexture = ReadFloat
ScaleNormalmap = ReadFloat
end function
function LoadDecal
ID = ReadInt32
Type = ReadInt32
TextureCount = ReadInt32 ' usually 1
For i = 0 To TextureCount - 1
Length = ReadInt32
TexturePath(i) = ReadString(Length)
Next
Scale = ReadVector3
Position = ReadVector3
Rotation = ReadVector3
CutOffLOD = ReadFloat
NearCutOffLOD = ReadFloat
OwnerArmy = ReadInt32
end function
function LoadDecalGroup
ID = ReadInt32
Name = ReadString
Length = ReadInt32
Data = ReadArray(Of Int32)(Length)
end function
function LoadProp
BlueprintPath = ReadString
Position = ReadVector3
Rotation.vX = ReadVector3
Rotation.vY = ReadVector3
Rotation.vZ = ReadVector3
ReadVector3 ' unused scale
end function
function LoadMap
If (ReadInt32 <> MAP_MAGIC) Then return
FileVersionMajor = ReadInt32 ' usually 2
Unknown3 = ReadInt32 ' always EDFE EFBE
Unknown4 = ReadInt32 ' always 2
ReadFloat ' Map Width (in float)
ReadFloat ' Map Height (in float)
Unknown5 = ReadInt32 ' always 0
Unknown6 = ReadInt16 ' always 0
Length = ReadInt32
PreviewImageData = ReadBytes(Length)
FileVersionMinor = ReadInt32
Width = ReadInt32
Height = ReadInt32
HeightScale = ReadFloat 'Height Scale, usually 1/128
HeightmapData = ReadArray(of Int16)((Height + 1) * (Width + 1))
TerrainShader = ReadString 'Terrain Shader, usually "TTerrain"
TexPathBackground = ReadString
TexPathSkyCubemap = ReadString
TexPathEnvCubemap = ReadString
LightingMultiplier = ReadFloat
SunDirection = ReadVector3
SunAmbience = ReadVector3
SunColor = ReadVector3
ShadowFillColor = ReadVector3
SpecularColor = ReadVector4
Bloom = ReadFloat
FogColor = ReadVector3
FogStart = ReadFloat
FogEnd = ReadFloat
HasWater = ReadByte
WaterElevation = ReadFloat
WaterElevationDeep = ReadFloat
WaterElevationAbyss = ReadFloat
LoadWaterShaderProperties
Count = ReadInt32
For i = 1 To Count
LoadWaveGenerator
Next
ReadString ' always "No Tileset"
Count = ReadInt32 ' usually 6
For i = 1 To Count
LoadLayer
Next
Unknown7 = ReadInt32
Unknown8 = ReadInt32
Count = ReadInt32
For i = 1 To Count
LoadDecal
Next
Count = ReadInt32
For i = 1 To Count
LoadDecalGroup
Next
ReadInt32 'Width again
ReadInt32 'Height again
Dim Length As Int32
Count = ReadInt32 ' should always be 1
For i = 1 To Count
Length = ReadInt32
If (i = 1) Then
NormalmapData = ReadBytes(Length)
Else
CurrentOffset += Length
End If
Next
ReadInt32 'always 1
Length = ReadInt32
TexturemapData = ReadBytes(Length)
ReadInt32 'always 1
Length = .ReadInt32()
WatermapData = ReadBytes(Length)
Length = (Width / 2) * (Height / 2)
WaterFoamMap = ReadBytes(Length) 'Obviously not used.. each byte is 00
WaterFlatnessMap = ReadBytes(Length) 'Obviously not used.. each byte is FF
WaterDepthBiasMap = ReadBytes(Length) 'Obviously not used.. each byte is 7F
TerrainTypeData = ReadBytes(Width * Height)
If (FileVersionMinor <= 52) Then ReadInt16 'always 0
Count = ReadInt32
For i = 1 To Count
LoadProp
Next
end fucntion
This post has been edited by HazardX: Feb 21 2007, 21:41 PM
Posts: 221
Game: