Read original data content instead of hardcoding it

[simonsan]

I see in the source files many hardcoded values of the original game mechanics instead of reading it from original data. To circumvent any legal issues you should try to use the .dat file as a data source or export it to a format usable by your engine (e.g. TOML, YAML, ... ).

You could for example use genieutils to extract the data and store it within a nyan container format - just as an idea. ;-)

Regarding e.g.:
https://github.com/puzzlepaint/freeage/blob/master/src/FreeAge/common/type_stats_data.cpp

Cheers

[MaanooAk]

I start by stating: I'm have no experience or knowledge on the subject.

You can't reverse engineer our games to access the assets or otherwise do things that the games don't normally permit in order to create your Items.

Do names and "values" of the units count as assets? In any case, the values are not reverse engineered, they are provided by the game (in the tooltips) and random sites on the internet (which actually make money by having ads next to the "values").

These are some comments for discussion, I may be wrong.

@simonsan Is there a specific section of the document you are referring to?

[MaanooAk]

@simonsan Could you also tag (if you know) someone from the OpenAge (or other projects) project who probably has researched the matter.

[puzzlepaint]

Thanks for bringing this up, it is obviously very important to me that we are on the safe side regarding legal issues.

Like @MaanooAk, from reading over the Game Content Usage Rules quickly, I however can't see a clear statement regarding game logic values, maybe I overlooked it?

If anything, "You can't reverse engineer our games to access the assets", sounds to me like, the less we access the original game data, the better.

However, in general I suppose that the game logic values may be covered by copyright - unfortunately, I know little about this.

However, I suppose that the WK mod used on Voobly might have the same issue. I think that they distributed the HD edition's balance changes as a mod to the original game. So, it seems like it must be legally doable.

[simonsan]

@heinezen Could you elaborate on this matter please. ;-)

[puzzlepaint]

To also elaborate a bit on the following: "You can't reverse engineer our games to access the assets" actually sounds to me like accessing even just the graphics data (which is stored in a proprietary format) would be forbidden. However, in practice, there are lots of external programs that somehow access game assets, including this data.

My impression (which might be totally wrong) is that according to the above-mentioned content usage rules, Microsoft would have the right to take down any such program that it is not happy with. But I guess that given how such modifications have massively contributed to keeping AoE alive, they probably don't usually enforce the rules.

Regarding the general question of whether Microsoft might be unhappy with a project such as OpenAge or FreeAge, before I released the first version of FreeAge, I contacted @TheJJ from OpenAge, and one of his statements was that they have Microsoft employees in Matrix/IRC and watchers on GitHub, and they didn't hear any criticism yet.

[simonsan]

@puzzlepaint In my personal understanding the difference is, that openage for example doesn't ship with hardcoded values from the dat-file but rather uses the converted ones. Like this it doesn't distribute copyrighted material and makes indirectly sure that people actually own the game.

But I think heinezen is the person that had to think most of it, so I think he would be able to give a more precise judgement here.

[heinezen]

Hello :)

The problem I see here is not so much related to the game content usage, but to the content distribution. The .dat file is technically a database that is copyright protected. If you have large parts of the database in your code, then you are distributing content owned by Microsoft. This can be a problem as companies are very petty about the redistribution of original content by fan projects. Microsoft has also done this in the recent past (example).

The solution to this problem is to either always access the original assets or access them once for converting them to another format. By doing this you also somewhat ensure that users need to acquire the original version of the game first.

[simonsan]

I think an important point to value here as well is that a game engine reimplementation is actually a replacement for the original game and not something like a mod or anything in addition or something to the original like an online tech tree or something, which is actually in the interest of Microsoft to keep because of community being involved. The game content usage rules are on purpose spongy to use them the way it's needed.

We can revoke this limited-use license at any time and for any reason without liability to you (and if we think you are abusing this license or its intent, you may hear from us).

So it's better to not rely on their good will in the first place, which is why I opened this topic - with good intent to this project. 👍

[MaanooAk]

So then someone could just web-crawl the wiki? or parse this SFTtech/openage@12ab282 and everything is fine?

[heinezen]

The easiest thing would be to just parse the .dat file and use the values directly from there. The wiki is not guaranteed to be up-to-date, neither are the openage docs.

[simonsan]

So then someone could just web-crawl the wiki? or parse this SFTtech/openage@12ab282 and everything is fine?

That would be funny, but no. You would still have it hardcoded and distribute it inside an replacement for the original engine. That's not the same like a document, that is basically useless for playing the game because it's not an engine at all - but a document.

Furthermore as something is hardcoded the license for using that hardcoded stuff could be just revoked - doesn't matter from where you have it if it's hardcoded into files in your project. That wouldn't be possible if you use the values directly or parse.
Furthermore with parsing from the web you didn't find a solution to the ownership issue, that you need to make sure that someone isn't playing with an illegal copy and just uses your engine with the cracked material - which is the biggest issue to Microsoft i feel.

[puzzlepaint]

Personally I don't really see the difference between distributing the game data values in a text document or on pages like this, and distributing them in some source code (which is also a document), but since I'm unfortunately not knowledgeable in copyright matters, I will take your word for this :)

Furthermore as something is hardcoded the license for using that hardcoded stuff could be just revoked - doesn't matter from where you have it if it's hardcoded into files in your project. That wouldn't be possible if you use the values directly or parse.

But how does reading the values from the proprietary game databases work together with the condition to not reverse-engineer the game? If I understand that correctly, I feel like there's no way to use the data without violating these rules in some way or another. However, people commonly do that, and some of the projects even seem to be embraced by Microsoft (e.g., the current Capture Age overlay, which I guess reads the memory of the game process, is being used in official tournaments). Thus, perhaps the main thing to ensure is actually that the project is in Microsoft's interest and doesn't harm them in any way?

Furthermore with parsing from the web you didn't find a solution to the ownership issue, that you need to make sure that someone isn't playing with an illegal copy and just uses your engine with the cracked material - which is the biggest issue to Microsoft i feel.

The question of whether someone could play with an illegal copy of the game is not directly affected by this, I think. FreeAge already reads the graphics files from the original, so it does require the original asset files to be present already (regardless of whether it in addition reads the game data files or not). On the other hand, reading asset files is not a sufficient means of protection against using an illegal copy, since these files can be easily copied separately.

Unfortunately, I don't see any good way to ensure that games like OpenAge / FreeAge are only used with original copies of the game. Even if they implemented some kind of ownership check, malicious people could easily remove that from the source code again.

[simonsan]

Personally I don't really see the difference between distributing the game data values in a text document or on pages like this, and distributing them in some source code (which is also a document), but since I'm unfortunately not knowledgeable in copyright matters, I will take your word for this :)

It is one thing to compile knowledge into a Wiki. This is obviously tolerated because it aids the community overall around the official franchise. The other thing is to put values and gameplay elements into a new engine and then redistribute it while it could be easily seen to stand in concurrence to the official franchise. That might be less tolerated. For obvious reasons.

In plain language: This project makes itself vulnerable to DMCA takedowns without any need just by hardcoding the dat file values in the source code. By this you are depending on the good-will of Microsoft laywers that your project is not taken down or you are sued. I think this is as far as I can take it, other internals I'm not willing to share at this point. Hope you're doing well.

[puzzlepaint]

Yes, thanks for clarifying, from this perspective it makes a difference of course. I was only discussing the (layman's) perspective regarding copyright before, not additional other implications. Just to make this clear here, I don't plan to pursue the approach to hard-code units stats and similar game logic values any further, for the reasons presented above. But I would appreciate a clear discussion on what the right thing to do is here (and what must not be done), since it is vital to this project, and so that any potential similar pitfalls can be avoided in the future.

Since my other questions from above have not been answered yet, I however somewhat got the impression that both OpenAge and FreeAge purely depend on Microsoft's goodwill anyway, as even with the approach to access original game data / assets, they cannot do anything to avoid violating at least one end-user license term respectively game content usage rule? If that is true, then it seems that this alone is also a possible reason for Microsoft to take any of the projects down at any time?

(I will not develop this project any further until this is clarified (and if the concern above can be voided).)

[heinezen]

The most common approach to avoid these issues that most major engine reimplementation project choose a clean-room design for the implementation and not distribute any of the original data with the executable. openage does it this way, as do openmw, OpenRA, RPCS3, Dolphin and others. The implication is that the engine and game data have to be separate by design. So while these engines contain the functionality of the original games, they can work independently without the original content.

Since my other questions from above have not been answered yet, I however somewhat got the impression that both OpenAge and FreeAge purely depend on Microsoft's goodwill anyway. as even with the approach to access original game data / assets, they cannot do anything to avoid violating at least one end-user license term respectively game content usage rule? If that is true, then it seems that this alone is also a possible reason for Microsoft to take any of the projects down at any time?

Any of the community projects depends on Microsoft's goodwill, but most are tolerated because they do not decrease the value of the project they are selling. Having unit data displayed in the wiki is something very different than distributing the unit data with an engine (for free) to play the same game they are selling.