Bug#776540: game-data-packager: please add support for games working with gemrbd engine

[Simon McVittie]

On 04/10/15 12:27, Markus Koschany wrote:
> If Baldur.ini in GameData with size 905 and md5sum
> 2dc7b81c210ed13ea88f509c765d4b76
> 	is_valid: true
> 	install_to baldurs-gate1-de package
> 
> If Baldur.ini in directory with size 905 and md5sum
> 2dc7b81c210ed13ea88f509c765d4b76 AND Baldur.ini in directory with size
> 842 e5526c907bb56ec72f8e6cf4eb30fc2e Baldur.ini
> 	raise exception ("That doesn't look like a valid archive")
...
> The current workaround seems to be to rename those files and to append a
> language specific suffix. Then one has to declare another stanza where
> "file x" is installed as "file y". (e.g. dig.yaml)

Yes. The "filenames" are almost more like opaque identifiers for a
particular file than anything else: as far as g-d-p is concerned, you
could call them anything, but it's a lot easier for humans to understand
the .yaml if you call the file Baldur.ini_de than if you call it
2dc7b81c210ed13ea88f509c765d4b76.

The look_for and install_as options default to the g-d-p-readable name
of the file, so that in the common case where the file's content is
unique, you don't have to do anything else. For the less common case
where the file's content varies, you have to tell g-d-p how to install
it, and whether it should look for any additional filenames (but the
default is look_for: [name, install_as] so that is often unnecessary).

We could have a heuristic like stripping some "extension" automatically
when determining look_for and install_as, but we'd have to use something
that is unlikely to appear in a genuine filename. One possibility would
be one of the characters that are syntactically invalid on Windows,
which are:

    \ / : * ? " < > |

(<https://support.microsoft.com/en-us/kb/177506>). Obviously we can't
use / or \, because we need / for relative paths, and \ is the YAML
escape character. To minimize awkwardness in the YAML syntax, we'd
ideally use a character that isn't special in YAML, but unfortunately
that rules out all of the Windows non-filename characters except "<",
and I don't like the look of that.

I'd vaguely lean towards "?", which is vaguely reminiscent of HTTP
query-strings, is relatively obscure in YAML (unlike ":" which we use
all the time), and is only special in YAML if followed by whitespace; if
we constrain the variation selectors (version and/or language) to not
start with whitespace, then we don't need special quoting. So we could
say that mentioning

    doom.wad?1.1

in a file list implies the equivalent of this:

    files:
      doom.wad?1.1:
        install_as: doom.wad

unless a different look_for or install_as value is configured?

In the unlikely event that a game's data includes the special "variation
selector" character, you'd have to specifically write

  # yes, really this filename
  files: why this silly filename?.txt
    install_as: why this silly filename?.txt

which doesn't seem too onerous.

> Another problem: the English version ships
> 
> 	data/AREA1700.BIF
> 
> the German version
> 
> 	Data/AREA1700.BIF

g-d-p searches case-insensitively anyway, because the 99% case is that
the game was designed to run on Windows or DOS.

> I install the same file into the data directory
> (English version) or the Data directory (German version).

Does gemrb really require you to do this? If you install either language
into the other directory (I'd use all-lower-case and move the German
version), does it work?

> For the last issue symlinking would be an option but this is rather
> inconvenient when you have to process a lot of files like that.

Or if gemrb is case-sensitive, can you install everything into 'data',
and use a single symlink Data -> data to accommodate the German version?

    S