#timetocode | Explore Tumblr posts and blogs

timetocode · 4 years ago

Text

Plans for nengi.js 2.0

Hi, this is Alex, the people’s network programmer and developer of nengi.js. Let’s talk about the future.

I consider nengi 1.x to be complete. Of course there are always unfinished items of work -- I wish I had a comprehensive tutorial series on prediction for example -- but really things have been stable and good for a long time.

So as I look towards 2.0, there are no fundamental changes to the library in mind. Instead the future is about improvement, making things easier, and staying open to deeper integrations with other libraries and possibly even with other languages.

One area of intended improvement is the whole process around forming connections both on the client and the server.

On the clientside, client.readNetwork() or equivalent is invoked every frame as the mechanism that pumps the network data into the application. However, this pump also controls the network data related to the connection -- meaning that without spinning the game loop one cannot finish trading the data back and forth that completes the handshake. I’d like to redo this such that we end up with a more normal api, e.g. client.connect(address, successCb, failCb) or equivalent. This presents a clean flow with no ambiguity as to when a connection is open. It’ll also let the clientside of games be a bit tidier as they don’t need to spin the network in anticipation of a connection opening.

On the serverside the whole .on(‘connect’, () =>{}) warrants a redo. I have in mind a simpler api where a ‘connectionAttempt’ occurs, and then the user code gets to invoke instance.acceptConnection(client, greeting) or instance.denyConnection(client, reason) thus again providing a nice and clean exact line after which we know what state the connection is in (attempted => connected, or attempted => denied).

Another area is Typescript support and some positive side-effects thereof. Nengi has minimal typescript definitions, but I think the actual surface of each api class/function should be rewritten in actual typescript. This will be limited, as the actual inner workings of nengi are truly untyped -- it has its own crazy typesystem and fancy optimization of high speed iteration based on object shapes that I should stop talking about now before I accidentally write a dissertation.

Per adding Typescript support there will be a major benefit to Typescript and JavaScript developers alike which is the opportunity for some top tier intellisense. The nengi api is small and having some modest documentation pop up right as you type things like .addEntity would be awesome.

The other benefit (ish..) of formally supporting Typescript is that a few of the processes around how to integrate game logic and nengi could finally be strictly addressed. I used to favor a very laissez-faire approach to integration as I didn’t want to stifle anyone’s style… but as time has gone by it seems that the level at which nengi can be decoupled is not seen as powerful, and instead it just confuses people. I want a better newbie experience, and presenting things like “well you can kinda do anything” isn’t helpful. I wouldn’t necessarily limit nengi itself, and instead may supply this functionality as a layer, but I would like to suggest a much stricter boilerplate for topics such as associating game data with a connected client and any other spot where game and network get glued together.

On that note of making things less open ended, I am *considering* whether nengi should offer an Entity and Message etc as part of the api. Currently entity is a concept or an implied interface -- really it is any object. Too decoupled? Maybe something more explicit would be nice. We’ll see.

More advanced protocols/schemas are also needed in the future. There are a bunch of features that can easily come from having more options on the protocols, but initially I plan to skip over all of these features and just change the api in a hopefully future-proof manner. The plan here is to change things from protocol = { x: Int, y: Int, name: String } to something more like context.defineSchema({ x: Int, y: Int, name: String }). Initially these will do the same thing, but in the future more arguments will be added to defineSchema.

The eventual removal of types from nengiConfig is another dream feature that may or may not make 2.0 but is worth a bit of discussion. NengiConfig.js is that file where every entity, message, command etc is listed out. Removing this would require nengi to be able to explain *in binary* how to *read future binary* and is non-trivial. The benefit however is that the parallel building of client and server code would no longer be a strict requirement. In the end of course a client and server need to be built for one another, but if the relationship were less strict than it is now it may pave the way for eventual nengi clients that aren’t even JavaScript. To me this has always been a bad joke -- who would want such a thing??? But as the years have passed it has become clear that nengi is not just special for being JavaScript, but that it is actually competitive in performance and functionality with the type of technology coming out of AAA multiplayer gaming companies (send money!!). So this may not be a bad direction (though it is worth noting there are at least two other major changes needed on this path).

There would also need to be changes to the current ‘semver’-ish release cycle. As it stands currently nengi version numbers follow the rules of breaking changes on major release (1.0.0) non-breaking changes on minor release(0.1.0) and small patches on patch release (0.0.1). As the current version of nengi is 1.18.0 that means that I’ve managed to add all functionality since release without a single breaking change (send money?!). This is not easy. These new changes described above are deliberately breaking api changes. Given the work cycle that I’m on and the lack of funding, the most efficient way for me to work would be with breaking changes allowed and perhaps a changelog to help the users out. So 2.0.0+ may shift to this type of development, where the ‘2’ is just my arbitrary name for the functionality, and 2.1.0 is a potentially breaking change. Obviously no one has to join me over in the land of nengi 2 until it becomes more stable, but letting me do *whatever* will get everything done faster, which is more important than ever given my limited time.

In the category of “maybe 2.0 things” here are a bunch of other things I’d like to talk about too, but they’re too involved (and experimental) of topics to go into detail. Here’s a vague summary of things I’ve put R&D time into:

Experimentally rewrites of sections of nengi in Rust, C, C++ with integrations via n-API, wasm transpilation, and some in-memory efforts. Crossing the boundary between JavaScript and anything else has been problematic as a means of solving most obvious problems, but some less-than-obvious problems may yet warrant this approach. I would say that n-API is a dead end for most nengi functionality but has some merit for spreading sockets across threads. WASM, or specifically working on a continuous block of memory may have some promise but requires further R&D.

An advanced rewrite of the nengi culler based on spatial chunking (promising!).

A middle api between serving up interpolated snapshots and the nengi client hooks api. This would become a generic replacement for the entire nengi clientside api. Until further typescript support I’m going to leave this one alone as it is very likely that a naturally elegant solution will show itself in the near future.

Multithreaded nengi, specifically the spreading of open connections across threads and the computation of snapshots. True optimal CPU use is opening multiple instances, not giving more threads to an instance, but there are some uses nonetheless.

Multi-area servers that use spatial queries instead of instances or channels (for example creating multiple zones, but not making an instance or channel per zone, instead the client.view just goes to a different space with some spatial math).

So yeah, that’s the plan! Thanks for your support (send money)

https://github.com/sponsors/timetocode

https://www.patreon.com/timetocode

#nengi.js #javascript #multiplayer #game development #programming

2 notes · View notes