Big post on 0.9.1 progress & future plans

Anything else to do with MoM IME
User avatar
Implode
Site Admin
Posts: 433
Joined: Fri Feb 24, 2006 3:35 am
Location: Newfoundland, Canada
Contact:

Big post on 0.9.1 progress & future plans

Post by Implode »

Big milestone today... the server finally compiles again!

I'll back up a bit to explain where I'm coming from here. From the start of MoM IME I've made a point of trying to keep changes small. i.e. I make one small change, test it to make it works, then move onto the next one. If anything is a bit of a bigger change, then (where possible) I break it down into smaller changes and do it bit at a time. I work well like that, and you get good quality code because every small bit you write you immediately test it. And that's good morale, you make a change, try it, and see it working almost right away. Sure, once in a while you make a small change in one place that breaks something somewhere else, nobody is perfect... but you find those bugs and deal with them.

Unfortunately not everything can be split up into tiny changes, sooner or later something like Fog of War comes along and you have to change a lot of the fundamental aspects of the game and rewrite major portions of just about everything. When you start changing the underlying data structures, every time you attempt a compile Delphi is nice enough to throw up compile errors to you and show you where you need to fix something, but it will only find the first problem and halt, so you can't pick and choose the areas you want to work on fixing first. I hadn't appreciated up until now how much of a drawback that is... more on this below. So this is what I've been at quite literally for months - I changed the server data structures as necessary for Fog of War, and have been ever since dealing with the ripple effect throughout all the code. I *HATE* working like this, because you have masses and masses of changes to make and cannot test any of it, because the application as a whole won't compile until every last piece of code has been changed. Its terrible for my morale, since I've been months slogging through code changes without being able to run it, can't see any benefit from my efforts, and can't guess how much there is left to fix. The last major game I tried to work on prior to MoM IME got abandoned for this very reason - I came to a point where I had to change some very fundamental concepts of the game, and the ripple effect was so big I was just as well to trash everything I had done and start again. So I started on MoM IME instead ;)

However, I've got through it, the server compiles again... I'm sure it has literally 100s of bugs in it now, but I can deal with that. There's plenty of stuff I've not even written yet (e.g. it detects when you try to attack a city, but it doesn't work yet), again I can deal with those one at a time now. Now it compiles again, I can run it, find a bug, fix it and am able to retest it again so eventually I'll get through them all. Also I've not done any fog of war work on the client yet, although I expect that to take weeks to do rather than being months of work like the server (because the client still only has one copy of the map, buildings, spells, etc. data, so has not really changed all that fundemantally... whereas the server in a 4 player game now has 5 copies of all the above to deal with).

Now, as an aside, over Easter I was out at my wife's parents for the 4 day weekend and can only take my laptop out there, which I don't have my MoM IME development environment on, so I worked on something else for 4 days. I had another Delphi program I'd written - its really just a front end to running DgIndex, VirtualDub, OggEnc and MkvMerge for converting DVDs into MKV files (no, I don't then share them illegaly, I use it on DVDs I own to get rid of the horrid teleclining and interlacing effects they introduce using current DVD encoding methods, in order to make them watchable to my very picky eye :) ), but to give myself some credit, it does some extremely clever stuff cutting & splicing audio tracks which I couldn't find any other program which did so acceptably. The Delphi version worked, but it was already fairly clunky and there was extra features I wanted to add (like the ability to cut & splice segments of multiple VOBs together) but the Delphi version just wasn't well organized enough to make those kinds of changes to. So I rewrote the whole thing in Java, adding in a pile of new really useful features in the process. I got about 60% of it done in those 4 days (probably only working it 2 hours a day), and finished the rest within a week of getting home. Which is pretty good for a mid-sized app. The Java version is MUCH more organized and structured, easier to follow, and uses data structures I'd never be able to implement in Delphi.

But working on this has given be a bigger revelation. Using Eclipse to edit Java code, when you change some fundamental data structure, it tells you everywhere that needs fixing right away - not one at a time - so YOU can pick which places to fix first rather than the compiler instructing you what order to fix things. Had MoM IME 0.9.0 been in Java, the server changes would have been SO much simpler to make and I reckon I'd have done it in half the time.

So I was always planning to eventually move the MoM IME server to Java, mostly because the memory management in Java is SO much easier (not having to explicitly free objects). But my feeling now is I might push to do this sooner rather than later. So probably something around 0.9.2 or 0.9.3 will be the last version with a Delphi server.

By contrast, I'm happy with the client being written in Delphi. Although there's a lot of it, the code in the client is considerably simpler than the server so doesn't really have the same issues. In fact I've yet to find a decent graphics drawing API in Java, so for now, Delphi is still the best choice for the client.

There's still some places in client where it does more processing than I'd like. e.g. when you click a unit to move, the client works out where you can move to, then when you actually select where to move, the server repeats the calculation to verify you're not cheating. I've decided this needs fixing, the client is too "fat" at the moment with too many calculations in it - it should just be the presentation and graphics layer, and not have any game rules or logic in it.

I've started thinking about how to handle multi-threading issues in Java, and been reading up a lot on the subject, and fiddling with the classes in java.util.concurrent, and come up with my own derivative of ReentrantReadWriteLock that supports all the nice deadlock-checking that my Delphi locking library does. *If anyone has some real experience with Java concurrency, I'd love to hear from you and show you what I've done here to get another opinion on my approach since its probably THE most important thing that you just HAVE to get right*

So right now, my plans look like this:
1) Get 0.9.1 finished and released. Sorry until I get a handle on just how many bugs I have to fix to get it in some kind of presentable state, I can't guess how long this is going to take.
2) Finish porting the server XML editor into Java. I've already started on this, but I'll make a push to finish it since I don't want to leave too many things half done.
3) Between 0.9.2 and possibly 0.9.3 as well, fix a couple of the bad design decisions I made in 0.9.1, especially about whether you can see what's being built in an enemy city (you can't in the original MoM, so I wrote it like this, but it turns out it was a really bad idea and overcompilcated a lot of areas... I'm not going to explain why since this post is already the length of a small book!) Also I'm somewhat of the opinion now that where I made the decision to not include units in the Fog of War (i.e. when you move away from an area, you don't remember what units you saw there) actually made things MORE complicated rather than simpler, so I might change that too. Move the movement code and any other complex code I can find out of the client to make the client as simple as possible, and aim to have all the complex code in the server.
4) Move the server to Java for 0.9.3 or 0.9.4.
...and see where it goes from there :)

So hang in there, things are starting to move again, and I appreciate everyone's patience waiting for me to get 0.9.1 out.

Implode.
Hal9k
Posts: 3
Joined: Sat Jun 17, 2006 8:42 pm

Post by Hal9k »

Implode - simple as that, you're a hero. I so much appreciate your effords and the time you invest in this project.
Warder
Posts: 6
Joined: Tue Dec 26, 2006 3:11 am

Post by Warder »

Thank you very much for the update, it's interesting to read the things you discover about the development even though I'm not personally interested in programming myself. Looking forward to seeing what's in the future for MoM IME :)
Coyote
Posts: 21
Joined: Mon Jun 26, 2006 11:09 pm
Location: Austin, TX

Post by Coyote »

Brilliant. That's great progress. Looking foward to it!
ikanreed
Posts: 13
Joined: Wed Oct 18, 2006 4:41 am

Post by ikanreed »

Java has it's problems. Poor support for graphics is one of them. If you want support for existing graphics code with an eclipselike IDE and a java-like language, I couldn't recomend anything more than C# and Sharpdevelop. Getting functional managed directX code into play(assuming you have a background knowledge in directX and from the description of the project I can only assuming you're using delphi to hook the C interfaces in windows for directX 9.0).

If you'd like me to take a look at your code sometime to see how long a proper changeover would take, please tell me. I'm already somewhat interested in the mechanism you're using to render the map, as that seems to be the main framerate issue.

I'd wager you're probably rendering each square independently, which can produce a huge drag on performance.

My knowledge of delphi is limited to how it works at the assembly level, but I can learn syntax(which I understand to be similar in structure to VB, which I do know) pretty quickly.
ixitixel
Posts: 1
Joined: Sat Jan 27, 2007 1:04 am

Post by ixitixel »

ikanreed wrote:Java has it's problems. Poor support for graphics is one of them.
AFAIK there are OpenGL classes for Java and a Java Quake implementation using these classes. This should be enough graphics support for a MoM client. Personally I also like C# more than Java, because I also come from Delphi and C# has many features that are also available in Delphi (Properties, a VCL like class library, ...)
ikanreed
Posts: 13
Joined: Wed Oct 18, 2006 4:41 am

Post by ikanreed »

ixitixel wrote:
ikanreed wrote:Java has it's problems. Poor support for graphics is one of them.
AFAIK there are OpenGL classes for Java and a Java Quake implementation using these classes. This should be enough graphics support for a MoM client. Personally I also like C# more than Java, because I also come from Delphi and C# has many features that are also available in Delphi (Properties, a VCL like class library, ...)
Right, but those libraries suck(mostly because java is an overly forceful language), and end up having unusual system dependencies that take away all the advantage of using java in the first place.
User avatar
Implode
Site Admin
Posts: 433
Joined: Fri Feb 24, 2006 3:35 am
Location: Newfoundland, Canada
Contact:

Post by Implode »

ikanreed wrote:Java has it's problems. Poor support for graphics is one of them.
That's why I'm only planning to port to the server to Java, which has no graphics at all.

If there's ever a viable graphics library in Java then I'd port the client to it as well (don't care about using the existing code, by "port it to" I mean "rewrite all the code from scratch"). But until that happens, I'm happy. Its the server I really want to move to Java because some of the code in there is getting really complex now with handling Fog of War, and I just find Java code so much cleaner, principally due to memory cleanup being 10x easier, and being able to declare vars throughout a routine, rather than all in a big block at the top.

BTW, every image (that is, every map cell, every unit, every letter of text, every button, etc. etc.) is rendered by a call to Device.DrawPrimitiveUP (D3DPT_TRIANGLELIST, 2, ...). If you want to see the code, email me and I'll either give you access to the CVS server or email you the relevant bits of code.

Implode.
ikanreed
Posts: 13
Joined: Wed Oct 18, 2006 4:41 am

Post by ikanreed »

Implode wrote:
ikanreed wrote:Java has it's problems. Poor support for graphics is one of them.
That's why I'm only planning to port to the server to Java, which has no graphics at all.

If there's ever a viable graphics library in Java then I'd port the client to it as well (don't care about using the existing code, by "port it to" I mean "rewrite all the code from scratch"). But until that happens, I'm happy. Its the server I really want to move to Java because some of the code in there is getting really complex now with handling Fog of War, and I just find Java code so much cleaner, principally due to memory cleanup being 10x easier, and being able to declare vars throughout a routine, rather than all in a big block at the top.

BTW, every image (that is, every map cell, every unit, every letter of text, every button, etc. etc.) is rendered by a call to Device.DrawPrimitiveUP (D3DPT_TRIANGLELIST, 2, ...). If you want to see the code, email me and I'll either give you access to the CVS server or email you the relevant bits of code.

Implode.
Oh, man, no wonder it's slow. That's like several thousand texture transactions per frame. Yeah, I'll drop you an email.
User avatar
Implode
Site Admin
Posts: 433
Joined: Fri Feb 24, 2006 3:35 am
Location: Newfoundland, Canada
Contact:

Re: Big post on 0.9.1 progress & future plans

Post by Implode »

Implode wrote:Also I've not done any fog of war work on the client yet, although I expect that to take weeks to do rather than being months of work like the server
See, I said it wouldn't take me long, the client now compiles again too :) Only took me 4 days (at only an hour or two a day).

There's still lots of work, and even more testing & fixing bugs, I need to do on both the server & client before 0.9.1 will be ready. But I must say, having been working on the Fog of War changes for literally months now (since mid January!), it was some nice this afternoon to finally see the game start up looking like this:
Image
:D

Ok, that was after I clicked through 3 or 4 error messages, and ignoring the fact that by this time, the server had crashed (FYI that's why it says turn 999 and the odd player name)... but hey I'm delighted just to see it running again!

Implode.
Lupinedreams
Posts: 83
Joined: Tue Jul 25, 2006 6:40 pm

Post by Lupinedreams »

I love that the default player name is Mr. Blah.

That slays me :D
ikanreed
Posts: 13
Joined: Wed Oct 18, 2006 4:41 am

Post by ikanreed »

I can safely say that's impressive. it seems like v1.0 might only be a half year out at this rate.
Lupinedreams
Posts: 83
Joined: Tue Jul 25, 2006 6:40 pm

Post by Lupinedreams »

If only! *drooling anticipation*
User avatar
Implode
Site Admin
Posts: 433
Joined: Fri Feb 24, 2006 3:35 am
Location: Newfoundland, Canada
Contact:

Post by Implode »

I totally did my head in last night figuring out how it needed to handle units walking in & out of each others visible areas, especially for Heroes who always overcomplicate things! But I got all the changes done, and gave it a couple of tests and it all seems to work fine.

The list of things I need to finish off for 0.9.1 is rapidly shrinking. The only other really complicated thing left on it is sorting out combat & attacking cities.

So I'm now fairly certain of what will & won't be included in 0.9.1, and a reasonable date when I hope to have it released by. See here.

Also made a note of what I am leaning towards doing for 0.9.2.

Implode.
aggelon
Posts: 71
Joined: Tue May 16, 2006 6:58 pm
Location: FRANCE
Contact:

Post by aggelon »

Waouh !

Like said Hal9k "Implode - simple as that, you're a hero. I so much appreciate your effords and the time you invest in this project."

I can't say in another way !!!!

Thank you very much for your job and efforts !!!!!

We love you !
Locked