I realise this has been a very, very long time since I last released an update, especially considering on the outside it isn't terriblly different from 0.9.3. The only major new feature is that you can now set the Units Fog of War setting to "Remember as Last Seen". But thinking back over everything that's gone on since then, I can at least understand myself why its taken this long, and to me that's one key feature that's a really, really big deal. So if anyone's interested, going to give some background here :)
I was struggling by the end of 0.9.3, there were some significant issues with the way the code was in Delphi that couldn't easily be addressed:
- Memory allocation/deallocation in Delphi is really tedious, you have to do everything by hand, free everything by hand, be very careful not to leave references to freed objects lying around, otherwise you get "Access Violation" errors all over the place.
- That limits some things you can do. In particular while I managed to have the server remember n copies of the building, spell, city and terrain data (that is, it knows not only the "true" values, but it remembers how the city looked the last time player 1 saw it, and the last time player 2 saw it, which may be quite different). But the data structures for units were too complex to replicate n times. It was going to be impossible to allow units to be "Remember as Last Seen", and without that impossible to write any kind of sensible overland map AI.
- The data model was haphazard, with items declared here and there - collating it all into a single data store to allow saved game files would have been incredibly difficult, and was a feature people kept asking for.
- I had attempted to allow it to be multithreaded, so the server could process messages from multiple players at once. But once a lot of the code was done, I realised this was a farce - practically EVERY update requires locking the list of players, and EVERY update requires locking the map terrain. So, all that happens is the threads all sit locked while they execute and complete one at a time. Meaning I was putting a huge amount of effort into carefully ensuring all memory reads/writes were locked and threadsafe, with no benefit from it.
- I had people ask, interested to help on the project, but Delphi just isn't widely used.
2 years into this, I realised I was getting nowhere. I was spending 90% of my time writing mundane code to read/write data files, and read/write network messages, to match what Delphi did, and 10% doing something actually interesting and useful. Moreover, I was precisely copying over the locking code so it could still attempt to be multithreaded, even knowing it was pointless, because I knew I had to do *something* to stop thread collisions when network messages from different players are arriving in different threads, and I just didn't know any better way to do it. So I realised for all my effort I really wasn't going to be much better off than I was with the Delphi code, the conversion was a pointless effort but I felt the Delphi code was at a dead end too, so not knowing what else to do, I gave up and trashed everything I'd done on the Java port.
A good amount more time passed, during which time I started working a lot more with web services, CXF and Maven at my day job. What I really liked about web services was that you didn't have to write any of the plumbing code - you just write a WSDL/XSD schema detailing the inputs/outputs of each service, and the various plugins would automatically create all the Java stubs to match those data structures for you, and do all the converting of XML>objects and objects>XML for you invisibly, leaving you to be able to concentrate on only the useful bits.
That really got me thinking, why can't I do the same thing with MoM IME. With some digging, I found with an xsd2java plugin and JAXB I could create all the Java stubs for all the data objects, and read in the whole XML file in about 3 lines of code - previously, I'd been months slogging through code to parse the server XML. Then I started to think, why can't I do the same for the network messages, so I'm not spending ages writing code to push/read those down the network - I mean, I know I can't use web services directly because they're restricted that they're always invoked from the client-side (the server can't initiate a call out to the client). But why couldn't I take all the useful bits of this, and just write my own wrapping protocol around it that meant the calls could be bi-directional?
So, I tried it. I modelled all the multiplayer network messages (not the MoM-specific stuff, just like, Log In, Create Game, Join Game, Leave Game). Then had some "fun" trying to figure out how to open network sockets and shove XML messages down them in both directions, keeping the connection open between messages. But I got it to work. The light blub also went on about how I could solve the multithreading issues - it did't matter if the network messages *arrive* on different threads, if I just wrote something to pull them out of those threads and be *processed* on a single thread - then if only one thread is ever accessing the game data, I don't need to screw around thread locking it at all, just all the data is open to all of the code - and it made everything so blindingly easy.
For a Java solution it felt perfect. The issue was, since I didn't want to try to port both the server and client to Java in one hit (would have taken forever), I had to get the old Delphi client to talk in XML down the network. I didn't know if that was even feasible, I mean, Delphi 7 is outdated now, XML was around at the time but not in heavy usage. There's a component and libraries included for it, but I didn't know how workable it was. So getting this to work was interesting to say the least, but managed. It was some legwork going through every single network message converting the old protocol into XML, but just worked through this as I implemented the same parts into the Java server.
And, after various other delays, hiccups and so on, finally got there.
So, from here on, if people find significant bugs in 0.9.4 (i.e. crashes game) then I'll do bugfix releases for these like 0.9.4.1, 0.9.4.2, and so on. But majorly I'll start looking at porting the client over to Java for 0.9.5. Handling the graphics makes me shudder how I'm going to do that in Java, I really have no experience in that. But at least all the XML handling and data model and all the network messages are all already done before I even start :)
