Still alive :)
Posted: Wed May 04, 2011 10:13 pm
Hi all and thanks to everyone for your continued interest coming back here from the fact that there are still new posts :) I'm also glad I disabled new registrations so the place wasn't completely overrun with spam posts. Its been a while since I checked back here and I'm sorry for that.
Going back probably 2 years now, I had been doing alright with the Java port of the server code, but... as with a lot of programming, its only after you try something once that you realise where the pitfalls are and how it should have been done in the first place. Considering this was supposed to be a port to a nice clean codebase in a readibly available language with lots of people out there experienced in using it, I had 3 really major problems with it (ignore this bit for the non Java programmers!)
1) I had a massive overhead spending huge amounts of time defining data structures, and how those are either read/written from XML (in the case of the database) or read/written down network sockets (in the case of the client/server comms messages), instead of spending time writing actual useful code.
2) Multithreading is always a headache, I tried to solve this the same way I did in Delphi, by creating a list of sequential locks such that locks always had to be obtained in a strict order so you couldn't get thread deadlocks. I thought this would be useful and allow the server to be processing multiple messages at once, but after working with it for a while the reality was every kind of message was locking the same things so they would all process sequentially anyway, so it was a massive amount of work for no benefit... which goes back to (1) about wasting time on overheads and not writing useful code.
3) I went nuts on generics, because I hate typecasting, so I wanted the multiplayer engine for example to be able to do .getSessionDescription() and get a MoM session description back without having to typecast it from the general session description. At the offset that was a good idea, but by the time I had multiple classes linking to each other I had some type definitions that were 3 lines long with all the generics listed out, and it just turned into an overbearing nightmare trying to manage it.
So between being bogged down with the 3 of those, and finding other games to play, it just kind of ground to a halt. Without having a clear way to proceed I lost all desire to try to. Since then I've now got 2 years more experience with using Java at work, specifically around using web services. You can't use web services for client/server games because the initiations are always client->server, the server cannot push data out to clients which it simply has to be able to for games like MoM. I wanted to get all the best parts of web services (specially things like, that you can strongly type the message layer in XSD/WSDL files and using Maven and wsdl2java you can get then Java objects representing all your messages generated automatically without having to spend ages writing them, or coding how the serialisation works) but lose the one way push restriction. And finally figured out how I wanted this to all work:
a) Have a big XSD which defines the structure of all the network messages - then use an xsd2java tool to create all the objects from this automatically.
b) Use JAXB to handle all the serialising of those messages down the network connection.
c) Use xsd2java and JAXB to handle creating Java objects representing the database and reading the XML in.
d) Change the network layer to handle all messages for one session sequentially, eliminating nearly all of the multithreading issues.
e) Has the ability for games not to die if clients disconnect, and has saving and reloading games built it from the start.
f) Take the 0.9.3 Delphi MoM IME client and change it to talk down the network using the XML that my Java network layer now uses.
a, b and c together will eliminate 75% of the Java code I had to previously write by hand. I started playing about with this to see if what I wanted was possible, and once I got a+b actually working got rather excited over trying to do stuff on this again. So am doing so when I get some spare time. There's a lot to do, I don't have the network layer finished at the moment, let alone building any MoM code on top of it, so don't go expecting to see a new version appear suddenly. But did want to at least let everyone know I am starting to pick this up again :)
Going back probably 2 years now, I had been doing alright with the Java port of the server code, but... as with a lot of programming, its only after you try something once that you realise where the pitfalls are and how it should have been done in the first place. Considering this was supposed to be a port to a nice clean codebase in a readibly available language with lots of people out there experienced in using it, I had 3 really major problems with it (ignore this bit for the non Java programmers!)
1) I had a massive overhead spending huge amounts of time defining data structures, and how those are either read/written from XML (in the case of the database) or read/written down network sockets (in the case of the client/server comms messages), instead of spending time writing actual useful code.
2) Multithreading is always a headache, I tried to solve this the same way I did in Delphi, by creating a list of sequential locks such that locks always had to be obtained in a strict order so you couldn't get thread deadlocks. I thought this would be useful and allow the server to be processing multiple messages at once, but after working with it for a while the reality was every kind of message was locking the same things so they would all process sequentially anyway, so it was a massive amount of work for no benefit... which goes back to (1) about wasting time on overheads and not writing useful code.
3) I went nuts on generics, because I hate typecasting, so I wanted the multiplayer engine for example to be able to do .getSessionDescription() and get a MoM session description back without having to typecast it from the general session description. At the offset that was a good idea, but by the time I had multiple classes linking to each other I had some type definitions that were 3 lines long with all the generics listed out, and it just turned into an overbearing nightmare trying to manage it.
So between being bogged down with the 3 of those, and finding other games to play, it just kind of ground to a halt. Without having a clear way to proceed I lost all desire to try to. Since then I've now got 2 years more experience with using Java at work, specifically around using web services. You can't use web services for client/server games because the initiations are always client->server, the server cannot push data out to clients which it simply has to be able to for games like MoM. I wanted to get all the best parts of web services (specially things like, that you can strongly type the message layer in XSD/WSDL files and using Maven and wsdl2java you can get then Java objects representing all your messages generated automatically without having to spend ages writing them, or coding how the serialisation works) but lose the one way push restriction. And finally figured out how I wanted this to all work:
a) Have a big XSD which defines the structure of all the network messages - then use an xsd2java tool to create all the objects from this automatically.
b) Use JAXB to handle all the serialising of those messages down the network connection.
c) Use xsd2java and JAXB to handle creating Java objects representing the database and reading the XML in.
d) Change the network layer to handle all messages for one session sequentially, eliminating nearly all of the multithreading issues.
e) Has the ability for games not to die if clients disconnect, and has saving and reloading games built it from the start.
f) Take the 0.9.3 Delphi MoM IME client and change it to talk down the network using the XML that my Java network layer now uses.
a, b and c together will eliminate 75% of the Java code I had to previously write by hand. I started playing about with this to see if what I wanted was possible, and once I got a+b actually working got rather excited over trying to do stuff on this again. So am doing so when I get some spare time. There's a lot to do, I don't have the network layer finished at the moment, let alone building any MoM code on top of it, so don't go expecting to see a new version appear suddenly. But did want to at least let everyone know I am starting to pick this up again :)