Crossfire has *serious* problems ahead...

Send your ideas and suggestions here.

Moderator: Board moderators

juk

Crossfire has *serious* problems ahead...

Post by juk »

I feel i should point out the following *very* serious problem(s) for the future of crossfire:

I have just spent many hours trying to get to the bottom of what has broken the x11 client and i am *very* sorry to say i feel the problem is *far* more serious than i first though it would be!

Sadly, the 'common' interface for the client's has started to become undermined in favour of the GTK client only (i will explain how/why later). The original idea of having a common interface was so that new clients *could* be written/ported to new toolkits, OS's, etc *easily* - WITHOUT having to do more than implement several (OS/toolkit specific) functions. This meant that whenever things were changed in the 'common' interface, they would be reflected in *all* the clients *without* much need for them to be changed drastically (ie: the 'toolkit' specific client was *only* responsible for display, user input, etc). This *was* a very sound idea, and whoever designed it this way, had abviously got a good idea of what they were doing (for a counter-example, look at the source for freeciv!).

Extract taken dirrectly from the source (x11.c & gx11.c):

/*
* This file handles mose of the windowing stuff. The idea is
* that all of it is in one file, so to port to different systems
* or toolkits, only this file needs to be updated. All windowing
* variables (display, gc's, windows, etc), should be stored in
* this file as statics.

* This file is largely a combination of the common/xutil.c and server/xio.c
* file. While I don't think this was a particulary great interface, the code
* was there, and I figured it was probably easier to re-use that
* code instead of writing new code, plus the old code worked.
*
*/

/* Most functions in this file are private. Here is a list of
* the global functions:
*
* draw_color_info(int color, char*buf) - draws text in specified color
* draw_info - draw info in the info window
* end_windows - used when exiting
* init_windows - called when starting up
* load_images - creates the bitmaps and pixmaps (if being used)
* create_pixmap - creates a pixmap from given file and assigns to
* the given face
* create_xpm - as create_pixmap, but does an XPM image
*
* draw_stats(int) - draws the stat window. Pass 1 to redraw all
* stats, not only those that changed
*
* draw_message_window(int) - draws the message window. Pass 1 to redraw
* all the bars, not only those that changed.
*
* draw_look, draw_inv: Update the look and inventory windows.
*
* NOTE: create_pixmap and create_xpm can be empty functions if the
* client will always use fonts - in that case, it should never
* request Bitmap or Pixmap data, and thus not need the create
* functions above
*
* Only functions in this file should be calling functions like
* draw_stats and draw_message_window with redraw set - functions
* in other files should always pass 0, because they will never have
* the information of whether a redraw is needed.
*/

Notice how it even *had* the idea of writting true 'rouge-like' (text-only) iterfaces etc!

I have looked into the client-side code quite seriously now, and as far as i can tell it *would* take very few man hours to write a new client using this clever (coding) interface... whether it was written in: raw X11R6, GTK, GDI, Direct-X, or any other toolkit/windowing system that was written to work/compile with C. I am very sorry to find that this is no longer the case, and the situation *will* only become worse over time - if this is not recified very quickly!

You may be asking yourself what the big problem is... I shall explain what *will* keep happening if this 'common' interface is not maintained:

1. Someone will write a new client, unable to use the features of the 'common' interface, they will copy large sections of the GTK client into their code.
2. The GTK code will get updated in parallel with server chages (as has just happened!).
3. The other clients will no longer work properly, until they copy the new code from the GTK client.
4. Eventually they will not be maintained, as the x11 client appears to be now, and they *will* become abandonware.

This scenario will happen *repeatedly* and eventually this code 'folking' will mean only the GTK client exists - leaving the future of cf down to the maintanance of GTK. You may well say "the GTK library is open source and will be maintained, so what is the problem?" - if you have been coding as long as i have, you will know this is simply not the case, and that libraries get obseleted and new (majour) versions are released to reflect hardware changes etc... This has happened repeatedly with comercial and non-comercial libraries and even X11R6 and the windows GDI libraries will go, but they have proved themselves and outlasted a lot of libraries over the yearsi (belive me!). For example, the cf cleint currenly uses v1.* of the GTK libs, and GTK2 *will* eventually take it's place, leaveing unmaintained v1.* code, and an *awfull* lot of work to eventualy convert the code, if not a total rewrite! For example:

Extract taken dirrectly from the source (gx11.c:void draw_info()):

/* draw_info adds a line to the info window. For speed reasons it will
* automatically freeze the info window when adding text to it, set the
* draw_info_freeze variable true and the actual drawing will take place
* during the next do_timeout at which point it is unfrozen again. That way
* we handle massive amounts of text addition with a single gui event, which
* results in a serious speed improvement for slow client machines (and
* above all it avoids a gui lockup when the client becomes congested with
* updates (which is often when you're in the middle of fighting something
* serious and not a good time to get slow reaction time)).
*
* MSW 2001-05-25: The removal of input from the text windows should
* work, and in fact does about 90% of the time. But that 10% it
* doesn't, the client crashes. The error itself is in the gtk library,
* to hopefully they will fix it someday. The reason to do this is
* to keep these buffers a reasonable size so that performance stays
* good - otherewise, performance slowly degrades.
*/

I would like to add that (in this case) it is NOT an "error itself is in the gtk library" (in this function at least), as i have disproved this by rewriteing this code to use terminal output (along with many other functions), in the hope of tracking down the reason the GTK client leaks 'X-server resources' (not a normal mem-leak), BUT i have not yet found the reason and it could well be and "error itself is in the gtk library" - in a totally differnt part of the code!

Now consider what would be the scenario if the 'common' (codeing) interface were maintained:
1. Someone will write a new client, useing the common interface to do most of the work (ie: only toolkit, window system or even terminal specific stuff).
2. The common iterface is updated, to add new features etc (such as the larger than 32x32 face stuff).
3. ALL the clients continue to work *without* the need for any copying of code from another interface, and most likely without the need for further alteration of eash individual client's code (assuming they are open-source, unlike DX...).

Now i shall explain the problem i have found:

The reason the x11 client has broken, is NOT because of it being unmaintained etc, it is simply because the common interface has not been properly used to add the large (>32x32) face/tile support... The x11 client will need major changes to add this feature, as it has no knowledge of this, eg:

Extract taken dirrectly from the source (gx11.h):

typedef struct {
void *icon_mask, *icon_image;
uint16 icon_width, icon_height;
void *map_mask, *map_image;
uint16 map_width, map_height;
void *fog_image;
} PixmapInfo;

Extract taken dirrectly from the source (map.c:void gtk_draw_map()):

if (the_map.cells[mx][my].tails[layer].face &&
pixmaps[the_map.cells[mx][my].tails[layer].face]->map_image) {
...
gdk_draw_pixmap(mapwindow, mapgc,
pixmaps[the_map.cells[mx][my].tails[layer].face]->map_image,
src_x, src_y, x * map_image_size, y * map_image_size,
map_image_size, map_image_size);
...
}

This could be fixed in the x11.c file, but this *will* result in the type of client 'folking' scenario i described above, and i *could* do it with only a few man hours of coding time (not an easy fix, but if the x11 client used the same structure elements, set properly, along with this test: it *would* work). BUT, why bother? This will only fix the x11 client until the next server/GTK client (paired) update is done, and the x11 client is doomed to either constant maintanance or abandonware... there is very little point in wasting valuable time on a problem which *could* have been avoided if care had been taken to maintain the 'common' (coding) interface, as the original authur had designed. Looking at this code, it *would* have taken very little effort to alter the x11.c code to use large face/tile support, if and only if the 'common' files had been updated properly.

This is only the begining of problems for future and current cf clients, and i know which scenario i would like to see happen, and if the ultimate aim of CF is to get to the stage of a single client relying on a single library such as GTK: please remember what i said here in 5 years time! I *have* seen, and suffered myself from, this sort of disaster before. I belive the original designer of the 'common' (coding) interface had great foresight and tried very hard to avoid this type of scenario!

Again, as with my last post on the metalforge update, i am sorry if i sound very 'to-the-point' and sorry if you read this as a non-coder (please don't get confused with user-interfaces and coding-interfaces etc!). Please feel free to copy this to the IRC channel, or post it to a more suitable forum (i am happy to explain futher the exact x11 problem and possible [temporay] fix, but see little point in making this post even larger!). Also i am sorry to point this problem out, and to the person who wrote the code i question here - don't feel bad about it, this is only the begining of future problems, and i only wish to see cf continue and thrive without ultimate disaster...

Thanks - Juk :)
Lauwenmark
Junior member
Posts: 111
Joined: Thu May 15, 2003 9:27 am
Location: Sélentine, I. Pref. Occ.

Post by Lauwenmark »

(Note: I moved your message to the Suggestions section, as it wasn't specifically related to the metalforge server)

You got a very interesting point indeed. Maybe it is time to rethink the common interface in a larger perspective ?

In my opinion, the common interface should be able to handle the following things:

Network protocol: the common lib should provide a layer of abstraction between the network protocol and the rest of the code. Would probably take the form of function wrappers, bound from the 'interface' part.

Raw data handling: I'm mostly thinking about image caching / data supply. The 'interface' part should be able to make something like a call to 'getRawImageData(pictureid)' without caring more about anything.

Configuration stuff: Loading/saving the configuration data.

Any suggestion ?
Au Nom de Son Auguste Majesté,

Lauwenmark Kadensanni Hento Akkendrittae
Général en Chef de l'Armée de l'Ouest.
Ryo
Forum Fanatic
Posts: 752
Joined: Mon May 19, 2003 9:16 pm
Location: Paris, France

Post by Ryo »

Reading your post, Juk, i agree on the need of a common library shared between clients, as it was (apparently) supposed to be.

Now a question comes to my mind. If i'm wrong in my assumptions, just tell me, and forget the rest of the comment :)

From what I understood by reading the forums, large image support (the main issue right now) has been planned since sometime, and actually implemented in the GTK client since a few months.
So why didn't someone at that time think about the X11 client?
I'm not accusing, blaming, whatsoever. Just wondering if X11 isn't already half abandonware.... (and I do not judge whether or not it should be that way.)
I remember some time ago mails on CF-devel about that large image support (granted, in the Java editor), so I don't think that support was developed in a secret place, either.

Not trying to flamebait, either, just wondering...

Just my 2 cents of € :)
Django

Admin Problems

Post by Django »

Juks Brother again. From juks comments i can see that there is no real structure throughout CF with respect to coding new udates into either the clients or the servers, otherwise why would this happen? I would like to ask a very important question who udates CF. It seems to me that there are only a few people working on it. Why if there was any real chain of command where all people could contribute would there be such problems? Who over all is incharge of the updates, if there is any such person. I seem to think that organistion is the main problem here coders on GTK seem not to realise the problems they are causing. Crossfire should be able to be used on all OS as juk has pointed out. NOT just on the OS of the couple of people who carry out the coding as has been the problem with the latest update on. They just havn't. I use DirectX client (with disfunctional graphics) at the moment and cant use winGTK despite what people say its unstable and crashes out. I think the updating should be released in a new version of CF 1.6 in both the server and client('common' interface) not as CVS then it can be ported to other OS. If anyone wants to correct anything please do i'm tried and talkin crap seems to make sence now but might not do later :)

Thanks Django
Avion
Senior member
Posts: 301
Joined: Tue Jun 03, 2003 1:16 am
Location: Canada
Contact:

Post by Avion »

I think that instead of being surprised by the nearness to disaster it would be more appropriate to be surprised that anything works at all - it only works because of some hard work by a few people. This isn't a problem unique to CF either by the way. It is hard to coordinate things. Now there are a few things that could be done to make things a bit easier.

The most valuable thing I think we could have would be to be able to search the development mail list. I am constantly surprised by the number of times the same things come up or major changes are not known. Large image support has been much discussed since release 1.3.0 (even read the docs - they encourage the development of large images) but everyone was surprised when large images finally made it into CVS arches it seems.
Online searching of the mail archives contents would be a really nice feature. The large image support was discussed on this mailing list (not in secret lab 42) and as far as I understand ALL developent must be vetted on this list. Since this is the case being able to search the dev list online is probably a good feature.

There is a Crossfire Documentation site targeted specifically for developers so that the documentation could be easily be read and commented on. Some of the simpler CF docs I have converted to structured text so that they can be both converted to HTML and differenced between the CVS and the online copy. This is ongoing, but if you have seen some of the documentation is years out of date.

This site is at http://crossfire.freezope.org

There are currently two wikis on this site as well(one for generating code and playguide related documents and suggestions, one for generating in game lore and legends). Wiki are very good for collaboration especially on things like play guides and feature proposals. Another idea would be to log any really good and important information from IRC to the wiki where people can go and either work it into a docuement (or tag it for editing) or usr it for reference to a problem that needs solving. In conjunction with setting up dates to work on things that need colaboration or knowledge transfer and with a searchable maillist archive - there could be a lot less wheel spinning going on.

I have also arranged to have put up on the crossfire sites Doxygen source code sites which allow the CVS code to be examined online. I will try to maintain this on a regular basis. There is alink to these in the news section of the realtime crossfire site, but soon I will customize the page templates a bit and make them function as parts of the document site as well. These are available at http://crossfire.real-time.com/code/ <server, client or editor>

There should be a TODO list in the arches as well as in the server and client - there are many half done graphics or server features waiting on new arches to make them work properly (off the top of my head raw materials for the object creation code and smoothing and player animation - also more work on classic set images). It would be good to have copies of the TODOs available (and maintainable) on the CF document site too.
Avion
Senior member
Posts: 301
Joined: Tue Jun 03, 2003 1:16 am
Location: Canada
Contact:

Post by Avion »

Ryo wrote: From what I understood by reading the forums, large image support (the main issue right now) has been planned since sometime, and actually implemented in the GTK client since a few months.
So why didn't someone at that time think about the X11 client?
I think it was simply missed or assumed to be fine (I assumed it was anyway) I personally have been testing on DX, GTKwin and GTKlinux. I hate using the X11 client...
OTHO I believe things like this have to happen to flush out bugs. If the code base has a lot of changes, updating metalforge may be one of the only ways to get folks to do a bug fix session... I for one am glad this happened before CF 1.6 was packaged.

Oh and BTW if it sounds flippant or irresponsible to say I assumed the x11 client was ok without testing it specifically - if you consider the time span involved and the number of comments made on this issue along with the documentation in the server source about this - it seemed pretty good bet that it was working - and even then I waited a few months and posted progress reports on the Dev list...
juk

The future of cf devel: long long long investigations...

Post by juk »

I have *finanally* got the x11 client to draw (inefficiently) maps which contain large images...

The function gtk_draw_map() is an example of one of *many* 'timebombs', that *will* go bang in the end!

I think this (current/cvs) comment sums up the foreseable future of cf devel:

/*After long long long investigations on why the multipart objects did
disappear when entering map view from right or bottom, scrolling
through whole server code, i concluded the following line should be
commented. If a multipart object was on a square outside of map,
containing the tail, tail may be cleared by following process and
so we end up with things like tower disappearance.
tchize@myrealbox.com*/

Notice the words: "long long long...", if things carry on this way, then more, and more, and more, time will be *wasted* on stuff like this, leaving less, and less, and less, time for actually adding things to cf!!!

I am *NOT* prepaired to give a patch for the x11 client, for the simple reason that i hope one day someone with more 'influence' than i have will try it also, and then maybe, just maybe, the current dirrection of cf devel will change...
HINT: It required a complete 'copy and paste' of the function: void gtk_draw_map() [<- see my earlier post on the loss of 'common' iterface], and simply substituting X-calls for GTK-calls (plus updateing many dodgy '32x32' initialized pixmaps/constants and adding/setting fields in the PixmapInfo struture), BUT it will probably take less work to write a new X11 client, than it will to get the mouse selection events working properly...

Sorry (again) to be so 'to-the-point', but it *will* be very sad to see yet another stable, well designed, well written, well commented, open source project, with a large user-base, go the same way as i have seen many go aleady (for example the 'nightmare' AI source in 'freeciv', and 'unstable/buggy' update of the 'frisk' project).

Good Luck - Juk :)
Avion
Senior member
Posts: 301
Joined: Tue Jun 03, 2003 1:16 am
Location: Canada
Contact:

Re: The future of cf devel: long long long investigations...

Post by Avion »

juk wrote: I am *NOT* prepaired to give a patch for the x11 client, for the simple reason that i hope one day someone with more 'influence' than i have will try it also, and then maybe, just maybe, the current dirrection of cf devel will change...
HINT: It required a complete 'copy and paste' of the function: void gtk_draw_map() [<- see my earlier post on the loss of 'common' iterface], and simply substituting X-calls for GTK-calls (plus updateing many dodgy '32x32' initialized pixmaps/constants and adding/setting fields in the PixmapInfo struture), BUT it will probably take less work to write a new X11 client, than it will to get the mouse selection events working properly...

Sorry (again) to be so 'to-the-point', but it *will* be very sad to see yet another stable, well designed, well written, well commented, open source project, with a large user-base, go the same way as i have seen many go aleady (for example the 'nightmare' AI source in 'freeciv', and 'unstable/buggy' update of the 'frisk' project).

Good Luck - Juk :)
There were so many things I wanted to say about this post but I suppose it would be pointless.

Did you post your findings to the developer list (I didn't see any...) or discuss it with some of the people working on the client? Did you get a negative response? Sending these comments on the issues you have seen with the client to the mailing list where the actual development is coordinated might be a good idea. Most of the developers working with the client probably don't even have time to read this forum and even if they do may not wish to comment on anything that isn't posted to the mailing list.
There are many fine talents who are not doing crossfire development - perhaps this is one reason that so many open source projects go the way you describe - many fine talents who would rather make points than contribute.
Ryo
Forum Fanatic
Posts: 752
Joined: Mon May 19, 2003 9:16 pm
Location: Paris, France

Re: The future of cf devel: long long long investigations...

Post by Ryo »

juk wrote:I am *NOT* prepaired to give a patch for the x11 client, for the simple reason that i hope one day someone with more 'influence' than i have will try it also, and then maybe, just maybe, the current dirrection of cf devel will change...
Am I reading your post correctly like (sorry, rough & blunt language, but best way to avoid any mistake) 'i won't give a patch because it was so much a pain to fix that CF's developers deserve to suffer to fix it too'?

If so i'm sad to see this attitude... If you really want to help fix CF, submit your patch for the benefit of all.

On the other hand, if i'm totally wrong, ignore my post :)
Lauwenmark
Junior member
Posts: 111
Joined: Thu May 15, 2003 9:27 am
Location: Sélentine, I. Pref. Occ.

Post by Lauwenmark »

A couple of comments...
Large image support has been much discussed since release 1.3.0 (even read the docs - they encourage the development of large images) but everyone was surprised when large images finally made it into CVS arches it seems.
(snip)
I'd like to recenter the debate a bit here. The question isn't about how surprized people were when the new changes came up; it is about how they made their way on a popular public server without (1)proper warnings and (2)enough beta-testing.
Also note that the discussion about large images was mainly conducted between developers, not players - This also could explain why information wasn't properly transmitted to everybody.

As for the developers docs, a good amount of effort has been put in it during the last few months. However, There are still many parts of the code that are poorly documented (if documented at all). We coders are to blame on that point - too often in the past we made changes without taking the time to properly document it.
i can see that there is no real structure throughout CF with respect to coding new udates into either the clients or the servers, otherwise why would this happen?
(snip)
True indeed. Note that it is an old debate between "freedom" and "consistency". There's no clearly defined testing/update policy for new additions - this already resulted in broken code showing up into the CVS. So far, no decision has been taken by the project administrators about that.
Crossfire should be able to be used on all OS as juk has pointed out. NOT just on the OS of the couple of people who carry out the coding as has been the problem with the latest update on.
True - but there are practical considerations to take into account. Most Crossfire coders are using Linux (or other Unix-like) systems. It is thus difficult to develop new things and be sure they'll work as expected on every system.
The portability problem isn't too hard to solve between different *nix systems - they're quite close to each other, and most of the time, only minor adjustments have to be done.
On the other hand, Win32 systems are a real portability bottleneck. A lot of system functions are different; GUI handling isn't the same; and so on. Win32 developers are a necessity to handle such problems - there are simply currently not enough working on CF.
I think this (current/cvs) comment sums up the foreseable future of cf devel:
Taking a specific source code comment out of its context for 'political' purposes is dangerous and unfair. Even in well-organized code, it is sometimes difficult to keep track of what every piece of code is supposed to do. Please stay on objective analysis and do not attempt to read into people's minds.
I am *NOT* prepaired to give a patch for the x11 client,
(snip)
What's your point then ? True indeed, the CF development process could be improved, and you're probably right to point out its most obvious flaws. But at the same time, you're not ready to try to help improving that - worse, you're giving moral lessons. You're saying something like I have a solution, but you're not worth it. I take that as a personal offense.
HINT: It required a complete 'copy and paste' of the function...
(snip)
This is serious coding, not a game of Guess-and-Win. If you have nothing more productive to tell, please don't waste our already limited time and let us focus on practical problems.
Au Nom de Son Auguste Majesté,

Lauwenmark Kadensanni Hento Akkendrittae
Général en Chef de l'Armée de l'Ouest.
Post Reply