Background: A Tale of Two Worlds

Note: I'm working on a mindblowing, super secret update. I don't know how long it will take so I will bridge the time in between with a series of more abstract posts, covering some of my current thoughts!

This blog post is about a new approach to capture and update the state of the ingame world in Citybound and what it makes possible.

The naive approach to state handling

In the old Javascript prototype of Citybound, all of the game state was stored in memory in a pretty straightforward fashion:

Each game object existed once, in its current state.

To update the whole simulation, I would go over each object and update it individually.

This had three big problems:

  • When I want to write a savegame, I have to wait until the current simulation step finishes to end up with a constistent view of the world in the save game.
  • When I update car A, I don't know if car B, which is in front of it, was already updated in this simulation step or not.
    • This leads to for example a wrong estimate of available breaking distance (because you're essentially comparing the future with the past).
    • In this case it's not that horrible (it just causes weird jittering), but for other interactions between game objects it might even lead to inversions of causality.
  • Once I update a game object, there is no way to see the previous version of it anymore.
    • For example, if it's payday and I apply the incomes of all the members of a household to its wealth, I can't find out in a following simulation sub-step, by how much the wealth of the household increased in this timestep - because I overwrote the old value to compare to.
  • The worst: if I tried to parallelize the simulation and I'm updating car A in one thread and car B in another, car A might try to get some information about B that is currently being overwritten by the other thread (imagine trying to read a number on a blackboard that someone is currently changing) - and you will end up with garbage information about B.
    • In the best case this leads to wrong behaviour.
    • In the common case this just crashes the game.
    • The only way to solve this would be to introduce so called synchronization locks, but they bring a significant overhead with them (imagine the blackboard guy always having to put up a sign "wait a sec, I'm rewriting the number" and you having to wait until he removes the sign again).

Introducing the "double buffer"

In the world of computer graphics there exists the concept of double buffering, maybe you've already seen a setting called like this in a game's graphics settings.

Double buffering is the solution to the problem that your graphics card draws to the screen pixel by pixel, but you only want to see whole, finished pictures (else you get a visible "tear" between the old and new pictures).

Note: This is exactly equivalent to the first problem stated above, that in a savegame you only want to see a finished simulation step, even though the simulation updates the world one object at a time.

So double buffering works like this: you introduce a second ("double") buffer to hold a picture for the screen, and then you always display one of them on the screen, while letting the graphics card draw (pixel by pixel) onto the other screen. When it's done, you simply swap the roles of the two buffers, showing the recently finished picture that's in the second buffer and starting to draw the next picture into the now hidden first buffer.

What if we apply the same concept to our simulation? What are the two buffers in our case?

In our case this means that we always have exactly two representations of the game world: the past and the future.

Very nicely following intuition, the following properties hold:

  • Updating the world means: determining the future based on the past.
  • After one simulation step ("time passes") the future becomes your new past and you start determining an even newer future (this is exactly the swapping of buffer roles).
  • Inside one simulation step, the past cannot be changed and you know for sure that it is correct and consistent, even when you determine the future in a parallelized fashion.

This truly makes parallelization possible.

Even better, this brings the following surprising advantages:

  • You can at any point in time create a savegame out of the past and even start saving it in parallel to the current simulation step.
  • It makes it much easier to test the simulation in a "does this actually lead to that" way.
  • For programmers: the simulation update looks much more functional on all levels - instead of willy-nilly mutating the current simulation state, you explicitly map an immutable past to a future in an almost declarative way.
  • If there is a bug in the game, where a correct past state results in an invalid future state or even a game crash, you can at least save the past state (as an emergency measure). Then you reload the game in this state and try fixing it until it doesn't crash and produces the correct future state.
    • This means that even if someone else finds a game-crashing bug, they can just send me their emergency-auto-savegame and I can debug exactly this case of wrong behaviour that they discovered.
    • That means that I can write the game with a "let it crash" mentality, where I don't try desperately to keep the game running even under invalid circumstances, trying to sweep them under the rug.
    • This makes a lot of code much simpler and leads to earlier discovery of bugs - while still keeping your most valuable savegame! (ideally)

The only drawback: you need exactly twice as much memory.

I would say: worth it.


Discussion on Reddit · on Simtropolis · on Something Awful
or follow @CityboundSim on Twitter
Donate with paypal
Every donation supports the development ♥

How I'm getting along

Hi, just a short sign-of-life kind of blog post.

C++ is pretty cool, everything I do feels more "proper".

Apart from that, I've got the pleasant problem that my day job became pretty damn interesting and demanding. Not as interesting as Citybound though - I still manage to work on it ~2hrs per day on average.

What I did so far:

  • wrote the basic part of compass, the core geometry module and in the process greatly simplified the code compared to the old one
  • wrote whiteboard, a helper library to quickly display 2D drawings, mostly used as visual debugging for the geometry code
  • sketched out how memory-mapped game state/save games would look in C++ (surprisingly natural!)
  • wrote lzy, my own helper library for iterators in C++ - since a lot of high-level code in the JavaScript version of Citybound made use of them and I want to keep the style
  • found inspiration for a simplification of the road network representation, should affect a lot of areas

What I'm currently working on:

  • implementing monet, a custom-tailored minimal deferred & physically based rendering engine that should allow for some pretty advanced and beautiful lighting shenanigans later on

The goal I'm working towards:

"Benchmark 1"

  • reimplement basic planning mode for roads
  • reimplement traffic simulation
  • stress test traffic simulation with huge amount of randomly spawned cars, see how far I can push it
  • also stress test renderer with amount of cars

That's all, I hope that gives you an idea! Speak to you soon!


Discussion on Reddit · on Simtropolis · on Something Awful
or follow @CityboundSim on Twitter
Donate with paypal
Every donation supports the development ♥

April Fools!

Don't worry, everything will stay in English. Kinda sad and funny that I have to write this disclaimer, there were too many people who believed me!

Now in German: Eine kleine Statusberichterstattung

Dear English readers: ever since I started rewriting Citybound in C++, I noticed how much more efficient something can be - just because you express it in a different language! Learning from this experience, I will now do all communication with the community, be it blog posts, answers to comments, code documentation and code comments in German, my mother tongue. I'm sure you will all appreciate the speed boost this will give me for developing Citybound.

... April Fools!

Wo ich jetzt stehe

Nachdem ich mich langsam mit C++ angefreundet habe und auch schon die erweiterten Funktionalitäten einsetze und mehr und mehr meistere, wollte ich euch einfach mal bescheid geben, was ich schon so gemacht habe und was so mein nächster Plan ist.

Wenn ihr mal auf Github schaut, werdet ihr sehen, dass es schon drei neue Module gibt:

Klar, die Projektnamen sind noch auf Englisch, aber all die Beschreibungstexte sind schon deutsch!

Kuckt sie euch gerne selbst mal an:

Bitte bei all dem beachten: ist noch ganz früh in Entwicklung und ich muss noch so einiges lernen in C++, also nicht wundern, wenn der Code komisch aussieht!

Der weitere Plan

Zunächst werde ich mich weiter auf die Geometrie konzentrieren, die ist halt echt das Kernstück von allem. Sobald das zufriedenstellend steht, ist das Hauptziel, Grafik, Benutzerschnittstelle und Spiellogik so weit wieder zu implementieren, dass der Planungsmodus wieder so geht wie ihr ihn von früher kennt.

Wie lange das dauert kann ich leider noch nicht einschätzen, aber ich halte euch wie immer auf dem Laufenden. Jetzt eben aber auf deutsch!

Liebe Grüße und bis bald!


Diskussion auf Reddit · auf Simtropolis · auf Something Awful
oder folge @CityboundSim auf Twitter
Donate with paypal
Jede Spende unterstützt die Entwicklung ♥

A Music Update From Dane

Greetings Citybounders, this is your composer speaking. While things progress in the programing side of the project, I thought I'd get you up to speed on how the music aspect is coming along. The trolly will be around with a refreshment selection of ginger ale or scotch momentarily.

I'll give a few paragraphs of exposition. If you want to skip this part, go to the bold and read from there.

The original plan was to write a few pieces for each season of the game (about 4 pieces per season), with each piece being approximately 5 minutes in length. Chris and I got to work drafting up some pieces and ironing out the contextual direction for the game. Since a soundtrack can have an immense effect on the success of a game, we were very careful with our choices at the start - writing about 8 pieces between us. Here is a playlist containing the initial tracks I wrote for the game. You might recognize some of them from Anselm's updates. (Once I get some links from Chris, I'll include his work as well. I don't like reposting people's work without their permission)

2015 came and Chris and I had other projects we needed to work on. Anselm began the arduous task of building the entire engine by which Citybound will run. It came with a lot of technical information that I didn't understand. I can program in Kontakt script, but that's it. We took the year to work on our other projects and I began to really sharpen up my skills by putting them to the grindstone of Abelton and unthinkable pounds of coffee.

February 2016. The Citybound crew of 3ish is still kicking and we've stayed in touch. Everyone is doing their own part of the project, and things are going as smooth as ever. I saw that Anselm was making rather impressive strides in his C++ programing/port, and I had an itch to write. Cue music update.

Here is a compilation of the new music I've written (and finished) since February. The music here is much more diverse, and reflects more of the Autumn/Winter side of the game soundtrack.

The direction of the music has gone the way of many simulation games from yesteryear. There is music in numerous genres from Jazz to House to Downtempo to Hip Hop, and it is all categorized in respective seasons. We have upped the music production to the point where each season will likely have more than 4 tracks each, with a total soundtrack length nearing the 2 hour mark. We are currently at an hour and 4 minutes, which made a midway point update necessary.

The end goal is to write enough music to create a vastly immersive, diverse playing experience as well as get great pieces out into the world. Nothing beats a game with snappy music, and we will do our best to deliver that to you.

I would like to request you, the community's feedback. Do you like where the music is heading? Does it sound good to you? What do you want to hear heading into the second half of the soundtrack? Can you tell I'm on 4 hours of sleep based on how poorly this is written?

Finally, I want to thank you guys for sticking around for this project. It's looking really fun, and I can't wait to get my hands on an alpha copy sometime.

Comments? Questions? Cusswords? Opposing viewpoints?


Discussion on Reddit · on Simtropolis · on Something Awful
or follow @CityboundSim on Twitter
Donate with paypal
Every donation supports the development ♥

After just two years, I'm starting properly!

During the whole development of Citybound I was half-knowingly waiting for the day where my selection of technologies would turn out to be inadequate. This day has come.

JavaScript and WebGL are amazing for what they are, but for what I want to do (create the best city sim game ever) they're too slow and too restrictive.

Instead, I decided now to do game development the tried-and-true old-school way and focus all my efforts on implementing & innovating, not on fighting language restrictions:

I will rewrite Citybound in C++.

C++

"chirp chirp"

...let me address all the surprise, rage, happiness and incomprehension that you might have, with a convenient question & answer format. Feel free to skip what you don't care about, since I go into a lot of detail.

But you believed in JS & WebGL so much! Why did you continue for so long and what made you finally doubt it?

  • Yes... I really wanted to believe. From the very beginning people told me that it's a stupid idea. But I didn't want to dismiss new technologies and not try new things just because of that. I needed to see it fail before my own eyes, after actually trying.

  • This happened now after implementing custom memory handling and then realizing in use that it's both too slow and a nightmare to program with (mostly because JavaScript doesn't have pointers and structs). My workarounds were beautiful, but in practice impossible.

  • After this relevation, I noticed a lot of other restrictions that I was playing down for myself and others (see advantages of C++ below).

  • But, I always told doubters that if JavaScript turns out to be the wrong tool, it would be at least a great prototyping language. This turned out to be very true!

So why C++? Isn't it horrible?

  • No, many (including me) are prejudiced about this. After taking a deep dive into C++11 for the last couple days and coding some first drafts, I have to say that it's a really nice language, allowing you to do horribly fast low-level stuff, while letting you package that in safe and nice-looking abstractions.

What advantages will C++ actually bring?

  • Much, much faster everything. This might make even my wildest goals of simulating true populations in the order of millions possible.
  • Inversely, this means that Citybound will run much smoother on even old and weak hardware.
  • Minimal amount of RAM usage and savegame size theoretically possible.
  • I can use existing, cutting edge and battle tested libraries for parts of the game instead of doing everything myself, for example:
    • bgfx for a modern graphics backend with all the newest features (WebGL is years behind that)
    • Eigen for natural looking and compute-optimal 3D math.
  • There is much more literature, examples and Stack Overflow questions available related to the obscure high-performance code that I'm writing. With JavaScript I was much too often the first one to ever do something.
  • Instead of hundreds of megabytes, the game's download size will be just a few megabytes (excluding only music).
  • The strict typing of C++ will probably prevent a lot of silly bugs early on and make it easier to find others.

What about cross-platform compatability?

  • Citybound will run as a native app on Windows, Mac, Linux and who knows what else! This is made possible by libraries for graphics, input, etc. which are themselves cross-platform (SDL 2.0 & bgfx).

  • Since I didn't want to publish Citybound as a webapp anyways (but as a webpage+browser Electron app), using C++ just means that the game will feel even more native on each platform (for example full multitouch gestures on trackpads/touchscreens)

What about moddability, tooling and open source?

  • My main reason for JavaScript was always that it is a very comfortable development environment and that a lot of people already know it.

  • Except some initial configuration hell (that I take care of for you), C++ also offers a great development & debugging experience, there exist very good tutorials and IDEs for it.

  • Since with native code I can protect a few select parts of the code much better, I actually feel comfortable open-sourcing all of the game-behaviour-defining code.

  • The game will come with a mod-kit that contains the source of the vanilla game in a perfectly set-up git repository. You would branch that into your own mod, modify whatever you want, run and try it and then publish it.

  • git's merge tools will help you to make your mod compatible with new versions of the game or even with other mods.

  • The way the engine is set up should allow something very close to hot-code-swapping where you can "operate on the living patient" for very fast development iteration.

  • Ideally I will find a solution for crosscompiling, so even though you only have Windows, for example, you can create working mods for all the platforms.

So..., will you finally use an engine now?

  • No, there still exists no engine that is optimized for the kinds of stuff Citybound needs (millions of actors...) and some essential things like advanced 2D geometry (for architectural stuff) is completely missing from engines.

How much will carry over from the old code?

  • The basic handling of game state, which is very compactly stored in RAM and can be dumped 1:1 onto the disk and then loaded again with almost no additional work required. This should make loading and saving super quick.
  • The geometry code should be pretty easy to be ported.
  • The user interface and intersection logic for roads and zones will also stay the same, so it only needs to be rewritten.
  • Graphics code will become slighly-lower level, but there's not much grahpics yet anyways, so I hope this will be fast.
  • Traffic simulation also "just" needs to be rewritten.
  • Economy and all the other gameplay needs to be written from scratch anyways.

How long will it take you?

I refuse to invent an answer to this question at this point. I know you hate me for it, but all of this is necessary and good. I will try to make more concrete promises like I did one month ago, as soon as they make sense again. Thank you for your patience.

You will give up on it! This is just the best proof that Citybound is nothing but Vaporware!

Never! Don't think that this was an easy decision to make, but rest assured that I will stay with Citybound until it is exactly what I want it to be, no matter how much time, effort and rethinking it will need!

You have even more questions, comments or concerns?
Let me know!


Discussion on Reddit · on Simtropolis · on Something Awful
or follow @CityboundSim on Twitter
Donate with paypal
Every donation supports the development ♥