Silkroad news, CREST project and NEKO archive extension

It has been while since I wrote something. So, some new information. We, people from ESRO project are hard working to provide you best gaming experience, although Joymax (also known as b0tmax or gaymax) is doing everything to screw with us. For example, in last update, they have changes OpCodes (identifiers) of packets. Thats is very nice of them, especially because they fucked up their own game as following screenshot illustrates, taken at Xian-65 server in this morning.



It is funny, is not it? No, not that pathetic char of mine, but that horse running around. It probably got startled by some Turk buffing his ass before starting Sroking, which was first bot to run with new OpCodes. At least some people say so. Hilarious!

Also, I have been working on garden today. This event is so rare, that my neighbours were worried about world's damnation. Poor things. I have to admit that I really enjoyed last few days (since Friday) which I spent working outside, without looking into my screen 15+ hours a day. Nice change :)

NEKO archive
So, what is it? Another troll? No, not at all. When I was younger and utter newbie to C/C++ (I was VB6 & PHP kid - hard to believe, is not it?) , I was playing with Burrows-Wheeler transformation and wanted to create super duper compression application. What a folly, I was stupid kid. But at last, I was not dreaming about creating super duper game like others did.

So, why am I writing about NEKO archive? It is not like I have travelled back into past and revived my old compression project I left to die back there.

There is only one reason - my Silkroad project requires it. Crests are little pieces of data, their length is only 256B. Even after converting them back into BMP, their size is about 1334 bytes (1338 bytes when processed by Silkroad client, which adds 4 empty bytes to the end of file) .

Having +- 1kB file is nice, it is not much. But main problem is, that there are thousands of these. Having this much of files on your drive is troublesome. Browsing takes ages, drive is fragmented, and work with these files is rather inconvenient.

Because of this, I have been searching for some archive to store crests in. I wanted to use TAR, however I have found out, that using it would be reTARded, since header for each file is 512 bytes big, filled with useless information or nulls (useless for me). Problem is not uselessness of data in it, but is size. It is twice as big as crest data size. Because this would be nasty waste, I have created much simpler archive format, which does not waste drive's space so much.

So, NEKO format is designated for CREST storing. I might extend it some time, to provide TAR-like functionality without wasting space. But I am not planning it now. I just don't feel like.

Because CRESTS are usually added and never removed (at least, ISRO does not remove them at all), my format is designed particularly for data incrementation.

Archive is structured into 3 parts, listed below:

1. archive header
2. colour palette
3. crest entries

More sophisticated description follows.

Size    Type            Name            Description
---------------------------------------------------
4       char            fileType        Specifies file identification, value NEKO is used.
4       long            fileVersion     Specifies file format version, value 0x01 is used.

1024    char            colorPalette    Specifies colour pallete used by Joymax. It is static member.

4       long            crestCount      Specifies crest count
varies  cstr            crestName       Specifies crest name, ASCII-string ends with 0x00.
256     char            crestData       Specifies crest data, its mapping to colour palette.

Well, that is all for today.

Silkroad maps (SMViewer) project - updated

Few days ago I wrote about my new project, about the SMViewer thing. I spent some time deciding whether I should use DirectX or OpenGL but then I decided that I will prefer OS portability over simple C# implementation for C++ ignorants.

I decided to use SFML window framework - it is simple, fast and nice made OOP, platform independent (*nix, Mac, Windows; what else do I need?) framework which has many other-language bindings (such as C#.Net and D).

My SMViewer engine does not support texturing yet, so I render only wire-network of map. Though it looks nice.

Screnshots






Video

Int. Silkroad CREST system revealed

Well we have revealed in it night - I have to express my thanks to Callum for borrowing me iSRO account and Rafa for packet and listening to me on MSN: "Thanks you very much!"

Basics

• ISRO does not use FTP, they use plain HTTP server for crest storing
• ISRO crest server does not have +Indexes (enabled directory listing), so you will always get, to following lovley message when trying to list crests. Wicked, is not it?

  디렉터리 나열이 거부되었습니다.
  이 가상 디렉터리에서는 콘텐트를 나열할 수 없습니다.

  In English (translated by great Google Translator) it says:

  Directory listing denied.
  This virtual directory can list the content.

• ISRO crest naming convention is same as in Chinese/Korean version
  

Packets
We (they, since I am not working directly on server emulator) had some unknown data in one of their packets. Here comes described data part of packet.

Size    Type            Name            Description
---------------------------------------------------
1       char            flag            ?
4       long            guildId         ID of guild
varies  scstr           guildName       Name of guild
4       long            crestIndex      Index of crest

Example with some actual data (packet was caught on Uranus server).

Name                    Value
---------------------------------------------------
flag                    01
guildId                 71 0B 00 00
guildName               0B 00 44 69 76 69 6E 65 45 6C 69 74 65
crestIndex              03 00 00 00 

Getting crest
Now, that we know structure of packet, we can assume how client gets crest from its (crest) server - no matter, whether it uses FTP or HTTP based storage.

However I have described structure of crest name in my Silkroad CREST project post, I will write it here again.

Name                    Description
---------------------------------------------------
type                    Type of crest (Aliance, Guild)
serverId                ID of server
guildId                 ID of guild
crestIndex              Index of crest

Knowing this, we will rebuild name of this actual guild's crest (using data from packet example and knowledge of Uranus server I).

Name                    Value
---------------------------------------------------
type                    G
serverId                187
guildId                 2929
crestIndex              3

Which takes us to name G187_2929_3.crb - easy, is not it?
Now, if we want (of course we want) to download this crest from ISRO, we have to use following url.

http://gdmark.joymax.com:15080/SRO_CREST/G187_2929_3.crb

Extra
As bonus, I give you list of servers with their IDs.

ServerId                ServerName
---------------------------------------------------
65                      Xian
74                      Aege
76                      Troy
94                      Athens
96                      Oasis
102                     Venice
107                     Greece
113                     Alps
114                     Olympus
132                     Tibet
134                     Babel
150                     RedSea
151                     Rome
152                     Sparta
156                     Eldorado
159                     Pacific
162                     Alexander
165                     Persia
166                     Zeus
174                     Poseidon
178                     Hercules
179                     Odin
180                     Mercury
181                     Mars
182                     Saturn
183                     Venus
187                     Uranus
188                     Pluto
190                     Neptune
191                     Hera (New)
194                     Gaia (New)
204                     Eos (New)
205                     Phoenix (New)
206                     Ares (New)
207                     Iris (New)
208                     Titan (New)
209                     Apollo (New)

Cheers!

Silkroad maps (SMViewer) project

Hereby, I am announcing it as my new project. I am going to use VC++ and DirectX (9.x), so project wont be Mac/Linux portable.

I have many reasons why I am doing this; one of them is that I would like to make IntroScript editor.

Surface map file (.m)
Brief description of m-map file, which contains surface height and surface texture mapping (with rendering options and brightness).

Header

Size    Type            Name            Description
---------------------------------------------------
8       char            type            File format type
4       char            version         Version of file format type

Content
Map block

• Each map block contains header with unknown meaning.
• Each map block contains data block with unknown meaning.
• Each map block contains 17x17 grid of cells.

Size    Type            Name            Description
---------------------------------------------------
6       char            header          Header, unknown meaning
2023    -               -               Cells
546     char            unknown         Unknown data block

Map cell

• Water begins on level 0.

Size    Type            Name            Description
---------------------------------------------------
4       float           height          Height of map cell
10b                     surfaceId       Surface texture ID
1b                      flag_6
1b                      flag_5
1b                      flag_4
1b                      flag_3
1b                      flag_2
1b                      flag_1
1       char            light           Lightness of texture

VC++ notation

struct MBlock
{
        unsigned char   header[6];
        MCell           grid[17][17];
        unsigned char   unknown[546];
};


struct MCell
{
        float           height;
        unsigned short  surfaceId       : 10;
        unsigned short  flag_6          :  1;
        unsigned short  flag_5          :  1;
        unsigned short  flag_4          :  1;
        unsigned short  flag_3          :  1;
        unsigned short  flag_2          :  1;
        unsigned short  flag_1          :  1;
        unsigned char   light;
};

Object map file (.o)
Brief description of o-map file, which maps objects into world.

Content
Group

Size    Type            Name            Description
---------------------------------------------------
2       short           objectCount     Amount of objects in group

Block

Size    Type            Name            Description
---------------------------------------------------
4       long            objectId        Object model ID
4       float           positionX       X coord (from left bottom corner of region)
4       float           positionY       Y coord
4       float           positionZ       Z coord
2       short           unknown         0xffff or 0x0000
4       float           theta           Angle on Axis Y
4       long            unique          Unique id of object, if same, object is not shown
2       short           unknown         0x0000 or 0x0001
1       char            regionX         X of region to be used as offset
1       char            regionY         Y of region to be used as offset

VC++ notation

struct OGroup
{
       unsigned short  objectCount;
};

struct OBlock
{
       unsigned int    objectId;
       float           positionX;
       float           positionY;
       float           positionZ;
       unsigned short  unknown_1;
       float           theta;
       unsigned int    unique;
       unsigned short  unknown_2;
       unsigned char   regionX;
       unsigned char   regionY;
};

Silkroad textdata/itemdata project

Silkroad saves information about game objects in text form - "CSV" (though it is separated by TABs, not by commas).

Textdata/Itemdata


Itemdata database cosists of 160 columns, which hold different values. (Why else would there be 160 columns, if they had same values, eh?)
Some columns are not described; that is because I am working on this in my spare time.

Columns description

A  # ?
B  # identificator
C  # resource identificator
D  # chinese name
E  # resource identificator (pair)
F  # resource identificator (name)
G  # resource identificator (desc)
H  # ? mall flag (sellable)
I  #
J  #
K  # ? {1 = equip; 2 = pets; 3 = etc}
L  # item type
M  # item subtype
N  # ?
O  # race {0 = china; 1 = euro; 3 = universal}
P  # item bonus {0 = none; 2 = sox}
Q  #
R  #
S  #
T  # flags { 0 = not storable; 1 = not storable; 128; 196; 255 }
U  # 0 = trade items; 1 = mall, 3 = rest
V  #
W  #
X  #
Y  # 0, 1, 129
Z  #
AA # price buy
AB # ? price repair
AC # ? price repair broken
AD # ?
AE # ?
AF # price sell
AG # requirements (1 = level requirement, 513 etc. are skill mastery reqs)
AH # requirements (pair; value)
AI #
AJ #
AK #
AL #
AM #
AN #
AO #
AP #
AQ #
AR #
AS #
AT #
AU #
AV #
AW #
AX #
AY #
AZ #
BA # urn of BSR - equipped
BB # urn of BSR - world
BC # urn of DDJ - icon
BD #
BE #
BF # stacking amount
BG # gender {0 = woman; 1 = man; 2 = unisex}
BH #
BI #
BJ # grade - ceil(grade/3)
BK #
BL # min. durability
BM # max. durability
BN # min. phy def
BO # max. phy def
BP # +val phy def
BQ # min. parry
BR # max. parry
BS # ?
BT # min. phy absorption
BU # max. phy absorption
BV # ? +val phy absorption
BW # min. block
BX # max. block
BY # min. mag def
BZ # max. mag def
CA # +val mag def
CB # min. mag absorption
CC # max. mag absorption
CD # ? +val mag absorption
CE # min. phy reinforce
CF # max. phy reinforce
CG # min. mag reinforce
CH # max. mag reinforce
CI # ?
CJ # ?
CK # ?
CL # ?
CM # ?
CN # ?
CO # ?
CP # ?
CQ # range x10
CR # min. phy attack (min. range)
CS # min. phy attack (max. range)
CT # max. phy attack (min. range)
CU # max. phy attack (max. range)
CV # +val phy attack
CW # min. mag attack (min. range)
CX # min. mag attack (max. range)
CY # max. mag attack (min. range)
CZ # max. mag attack (max. range)
DA # +val mag attack
DB # min. phy reinforce (min. range)
DC # min. phy reinforce (max. range)
DD # max. phy reinforce (min. range)
DE # max. phy reinforce (max. range)
DF # min. mag reinforce (min. range)
DG # min. mag reinforce (max. range)
DH # max. mag reinforce (min. range)
DI # max. mag reinforce (max. range)
DJ # min. attack rating
DK # max. attack rating
DL # ?
DM # min. critical
DN # max. critical
DO # hp recover amount
DP # - chinese
DQ # hp recover percent
DR # - chinese
DS # mp recover amount
DT # - chinese
DU # mp recover percent
DV # - chinese
DW #
DX #
DY #
DZ #
EA #
EB #
EC #
ED #
EE #
EF #
EG #
EH #
EI #
EJ #
EK #
EL #
EM #
EN #
EO #
EP #
EQ #
ER #
ES #
ET #
EU #
EV #
EW #
EX #
EY #
EZ #
FA #
FB #
FC # magical options unit count
FD #

Note: Columns AG-AW contain requirements (such as certain skill level, etc.). These columns are paired. Example and description shall be supplemented.

Silkroad CREST project

One night, I was thinking: "How does the emblem system work?". I could not sleep so I started to fiddle with Silkroad client and started off this this project.

Basics
Silkroad stores guild/union crests on stand-alone FTP server. There is no obvious reason for this implementation and I do not understand why they have done it in this way.

Another weird fact is that files are kept in single directory which makes it really difficult to load (takes long).
Following example shows list of some files from crest storage.

A10_1363_2.crb
A10_1363_3.crb
G90_350_5.crb
G90_350_6.crb

File naming

A10_1363_2.crb

File name written above can be split into these parts:

Name                    Description
---------------------------------------------------
type                    Type of crest (Aliance, Guild)
serverId                ID of server
guildId                 ID of guild
crestIndex              Index of crest

File data
Silkroad uses 16x16 pixel big 8bpp bitmaps with static color palette and no compression. Thus, stored crests on FTP server are 256B big.

Once you know this, it is simple to rebuild "original" bitmap from crest file.
You only need basic knowledge of BMP structure. BMP is composed of 4 logical parts, as is shown below:

header
meta
palette
bitmap (crest file content)

Crest file contains bitmap part of BMP. Since header, meta and palette are static, it is easy to rebuild bitmap:

static part (header, meta, palette)
dynamic part (bitmap)

Issues

1. FTP server is damn slow
2. Requests timeout on some occasions

Results

1. Crest downloader & converter (running)

   

2. Some crests from Korean Silkroad.