Topics I didn't touch:
- Ruby, pretty icky unless done right
- The woes of over-designing
- The woes of over-optimizing
Want me to write about these? Just ask. This was getting pretty long I could have gone on much longer.
Topics I find moot:
- But the community doesn't understand Ruby so therefore that's why nobody used Apollo?
I'd say this is true for newer users, and given the premise of my topic I'd argue that it was a bad move to use Ruby for plugins but there are a very limited amount of scripting languages that are suitable for things such as this. However though, I think people would adjust to the skill floor if they saw that it was worth it. It just wasn't worth it for people to do so. Even more advanced developers struggled with the design vision, myself included.
Apollo, regarded as one of the greatest pieces of Runescape emulation software, was created by Graham Edgecombe in 2011 and insistently since it's release has been pushed onto the community as the
source to use. Even if this is the case, given the current environment you are able to conclude that most public RS emulator sources do not use Apollo as their go-to platform to start a project. As popular opinion may voice, this seems to be a contradiction to what one may consider to be an attribute for one of the greatest pieces of Runescape emulation software to possess.
I'm here today to present to you an opinionated piece from my own perspective backed by logical reasoning of why I regard Apollo as a failed project, that with restructuring, could find a niche into actually being a good piece of Runescape emulation software. Personally of which I DO NOT
regard it as such.
The premise of this argument I'm presenting takes into account that because Apollo is regarded as one of the greatest pieces of Runescape emulation software to use, and is commonly recommended to people for use must mean that it is a good piece of software for the public domain to use.
Let's look at Apollo's, out of the box, strengths. The referenced repository can be found here
. This repository is considered the official Apollo source by Major and Graham. What I would consider a strength is a well thought out solution to a parameter of design that has to be implemented in coordination with other solutions to fundamentally solve the main problem which of course is in this case emulating a Runescape server.
- Strong and easy to understand design from a engineering perspective.
- Excellent use of concurrency when encoding Player and NPC synchronization packets.
- Well organized.
Now, I will state that the extent to which you correctly emulate a Runescape server is the main determining factor for how successful a piece of software will be. Let's see what they consider the strengths of Apollo and compare with my evaluation, I refer to this since it is headlined under 'What Apollo does different' which could be reworded as 'What Apollo does better':
Packet encoding/decoding has been split from the representations of the packets themselves. This allows the potential for encoding/decoding to go on in parallel and also allows multiple revisions to be supported. Currently 317 and 377 are both completely supported.
Update server support (JAGGRAB, ondemand and HTTP).
Packet handler chaining: this allows multiple plugins to be able to intercept a single packet and deal with it appropriately. For example, a quest plugin could intercept searching a bookshelf for instance, if the behaviour needed to change in certain cases.
- Parallel execution of player updating for multi-core machines.
One out of the four given 'prominent core features' I would consider to be key or strong features. Why?
1). Given the nature of the Runescape protocol with the delay between frames of time also known as ticks all data from messages received from clients have to be handled in an asynchronous manner to assure that no race conditions occur in the provided application because of how messages mutate models on a consistent basis. Asynchronously decoding and encoding messages isn't even responsible in the same context because even in this applied design a netty worker loop thread will asynchronously decode and encode messages when they are flushed to the socket channel. This is a pretty moot point, it's expected behavior and it's worded in a deceiving manner to make it seem like this is the responsibility of the emulation itself when rather the actual behavior is not in the manner it was described.
Yes, the famed 'multirevision' holy grail. As a personal note I have always up until the past year considered this to be a strong selling point with a piece of emulation software. My opinion on this changed when my philosophies of design changed. Designing features that detract, or even add too much complexity, to a problem that could be simply solved in a manner where the easiest efficient solution is implemented wastes time efficiency and leads to over designing. With the implemented design, I do not deny that it would be possibly for multiple revisions to be supported because the message layer is often revision to revision agnostic.
A majority of the packets in a RuneTek 2 (200-400) client are found in a RuneTek 3, 4, and even 5 client. This is because Jagex had no reason to really change their server architecture. If it isn't broken, don't fix it. I don't think it's commonly understood that Jagex was very cost oriented, a lot of their protocols are VERY bandwidth efficient. So to change a protocol that has worked for them in an optimal manner would cost money for no prospective gain. That's just common business sense.
However, lets take a peak at how the design for this is set up.
The way that the Apollo multi-revision support is set up is by registering message decoders and encoders which are responsible for decoding a message in a specific manner. The reason for this is because in order to inhibit packet bots, such as Aryan, Jagex randomized packet opcodes and streams so that the data between the client and server would be difficult to decipher. This is fine, except for the fact that behind the madness of this implementation you have to consider that you will have to create a single abstraction to decode and encode a message. If each abstraction is confined to a single class then you are writing up to two-hundred and fifty-six classes just for this purpose and these classes will have to be modified for every revision.
This causes a lot of boilerplate, boilerplate is seemingly repetitive code that shows up in various occurrences that seems like it could be confined into a smaller simpler manner. I would consider this to be one of Apollos major flaws, and frankly it's hard to engineer around this efficiently and I do owe this to the fault of Graham or the Apollo developers at all.
Under modern circumstances, we know today that there is a common protocol between various client revisions where incoming packets under certain circumstances are transcribed to their actual packet id and have the streams normalized. This data is present in RuneTek 2 clients in the packet constants, between revisions the packet ids are consistent and structured in a manner that logically makes sense.
Graham and his team could possibly not have known this at the time, but for future considerations especially in more serious development projects I'd consider the issue of packet decoding somewhat a priority to implement in an efficient manner as to help stifle cheating programs that may have the intention of harming your server. It's a very difficult engineering task, and even though we have this modern information it's hard to guess how these numbers are generated, by what method, and if this piece is reversible. I'm willing to wager it isn't.
2). Under my definition of the extent of how well a server is emulated I argue that in order to properly emulate Runescape and be a proper piece of emulation software this isn't anything that is extraneous under any circumstance so is a moot point. Serving assets also recently has been found to be distributed across ALL Runescape nodes because the cache for Runescape holds server sided configurations used in their implementation. This should be taken into consideration but is fairly moot under the premise that it isn't required to properly emulate the Runescape game however it sheds light on a method to solve solutions that you will come across when trying to organize assets.
3). This piece includes a critical flaw of design that is repeated across Apollo and bridges fundamentally into what I consider to be a major philosophical issue. I will touch up more on this below because I think it needs its own explanation and section.
These are issues I'd consider behind the scenes which may not be obvious to people except for more conditioned developers. Now the diction of this is important. Behind the systems in Apollo, under the definition presented above there is a conflict of interest for a target audience where the software released seems to cater seemingly to no public domain. These systems are not well explained and not obvious to people who may want to use this source for a project although they are marketed to being the essential features that put Apollo apart from any other source.
Well, what features are important to people? For this question I turn to what I observe to be great sources for public domain use. And personally I begrudgingly mean; PI, Matrix, and Delta. Honorable mention to Arios, I do feel as if their team deserves a great amount of credit for pushing the boundaries of what people have been able to emulate. These sources are popular and widely used across a popular spectrum of public Runescape servers is because they have either a strong way for people to construct content, or contain a lot of content. Now you may be turned off from the thought of me saying that a PI, Matrix, or Delta source have strong ways for people to implement content but they really do. Each have the tools to implement a wide array of functionality that best suites the server developer's interest, and some do it in a way that basic programmers can understand. Apollo does quite the opposite, but I think it's important to bring up history of Grahams developments, the internal client systems in place for this, and how this relates to game programming in general.
Graham, and I state this as a fact, has never publicly released a properly engineered content system. The definition of proper is loose, and I state this as fact because beyond Hyperion, even under the condition of well thought out design, no single server has been extensively used under the definition of the premise I found this argument on. None of his releases, besides his early ones, are widely used under public domain. If the content system was properly engineered, then there would be a wider sample of people using Apollo or his other releases beyond Hyperion as a platform for their project. This isn't the case.
The reasons for this are limited functionality. Tying in with the earlier topic of misdirected design, all releases beyond Hyperion that were originally written by Graham don't have a wide array of use. They have the limit basics, but not all of the basics. His Apollo release had core emulation features (player and npc updating, asset serving, walking, inventories, and banking) and that was it. Given that Apollo was recommended for public domain, and under the assertion that newer users may consist public domain it's impossible to expect for those users to implement the rest of the core functionality. And given the amount of recommendation, does it make sense to expect people to do this?
You may notice a missing note about the scripting system.
Well, lets transition into discussing why I consider the scripting system a moot system. This may be a sensitive subject with people and I have a lot of personal experience with developing systems such as this that I will incorporate with my analysis. This topic will also include, what I think to be, the worst critical design flaw of Apollo.
Apollo uses JRuby plugins to implement various functionality, each plugin is responsible for a specific content feature such as banking, mining, or woodcutting. The main purpose of this system was to move people out of developing content in Java and give people the ability to import plugins from various authors to create a custom emulation platform. The exposed functions to the scripting environment were extensions of methods in Java used for registering handlers to message pipes, or changing various parameters.
Aside - I'm laughing writing this because man, this was a flop.
The amount of snippets that are Apollo plugins, with the included consideration of the lack of popularity, is <1%. Each plugin that was released to the public suffered from a lack of consistency because of the lack of tools given to developers to do simple tasks. This defeated the entire purpose of giving the people the ability to import plugins from various authors and was a large oversight from Graham in his vision.
Dependencies for plugins were managed by the plugin loader, which helped give people the ability to write utility plugins or use common plugins to write in functionality for their plugin such as using the banking plugin during a quest line. The implementation of this idea was a disaster and another major oversight.
Apollo's design, as stated in their opening introduction, has a built in chain of command for propagating messages and handling them in turn responsibly. In turn this is noted to be a feature that sets Apollo aside. The design pattern for this is a chain of handlers or form of pipe. Coupled with this was a passed context which is responsible for optionally stopping the propagation of a message down the pipe. A logical layer is responsible for registering handlers to the chain, or pipe, with the partial knowledge of knowing how the messages are processed. I say partial because there are multiple layers that are responsible for registering handlers to the chain, none of which are assumed to communicate with each other or understand what the current state of the chain may be since there was no way to query this information. It would have been extraneous to give this functionality, and counter intuitive to the simplistic nature of this implementation. There in this lays the critical design flaw. The design of content was implemented in such a way that people can add or remove plugins from the emulation platform. Plugin functionality is expected to be isolated by design, with of course dependent plugins being apart of inline functionality. Therefore, since plugins are expected to be isolated by design and in this design the nature of how messaging handling is implemented it's impossible for plugins to understand what responsibility of functionality they encompass. Certain plugins may consume messages before they are propagated to an awaiting message handler in another plugin by the nature of isolation. This leads to issues of functionality of design where with this system you put the developer in a black box where in order to fully understand the system they have to understand all of the implemented plugins. This goes against the vision of design, and therefore is illogical.
As a programmer, I know since that point in time was four years ago, this is unacceptable. And expecting the public domain to develop under these conditions or even suggesting this was a valid solution is illogical.
The lack of functionality, although enough to turn most of the public domain away from using Apollo, isn't entirely present in most sources. It wasn't until recently that people concluded that the player 'configs' were actually the players game state. Method confided this in multiple people at a much earlier date but it hasn't been even considered to be an Incorporated system in any source. Technically there is a very limited amount of data structures that describe the player save state; those being the skills, inventories, and variables. I use the word variable to define a key entry styled data structure which maps a unique integer, the id, to an integer value. This structure encompasses data for quests, object states, item states, and configurations. Variables are used with the player, npc, and even item model. We can confirm this from Runescript leaks. An synonym for this is meta data. Although the structure is very primitive.
All of these structures can be appended to models in such a way where in an extensible manner you can append or remove them without changing fields by assigning them unique identifiers that are registered in the context of the game environment. This is critical for scalable development, however it wouldn't be minimally required for emulation development.
Under the intention of giving developers the power to have a wide array of functionality with plugins however, in Apollo I'd consider this a critical priority. I however did not think Graham had much of a game development background where he could implement these features in a logical manner. It's possibly he didn't consider it important. However it doesn't make sense to implement this vision, the vision of easily deployable plugins, and not see through through with it.
As a game programmer I think the perspective of priorities change especially when you have more insight to trying to implement systems that scale to fit the needs of what you may need to implement. If you limit yourself to certain nuances or do not give yourself enough options to where you can efficiently, in the terms of a raw coefficient of efficiency, then you are hindering your development and not being logical.
Grahams Apollo, and essentially his other releases, were very limited and were just essentially in the very essence of the word: frameworks. You cannot compromise on advertising how certain design visions are better than how other designs are implemented, and then suggest that people should use the source or even call it anywhere close to be one of the greatest pieces of emulation software.
Guys, keep using PI, Delta, and Runesource. THINK like a game developer.