After some experimentation and measurement, I proved an obvious fact today: the naive approach of just exploring all possible daily schedules grows out of hand in complexity, astronomically quickly.
For example: even in a tiny city that just consists of 4 places (shops or homes) with corresponding activities from my rural-village scenario, and connecting roads, I got over 80.000 daily schedules that would have to be evaluated to come up with a feasible lifestyle for a single person.
When I tried to do the same for a city with 9 places, it just ate my RAM. Such are the impossibilities that a guerilla-city-sim programmer has to face...
Does that mean that I'm just doing pointless excercises? No - trying stuff like this and seeing the grandiose scale of the problem I'm facing in front of my eyes helps tremendously - both in a philosophical, as well as a very practical sense. I can actually see how much work I force the computer to do, where, and why - and this fuels my intuition to come up with a better approach.
My next step will be to implement my first try at something smarter (based on simulated annealing, as was planned), informed by what I learned today.
One thing first: through elaborate sarcasm people often express serious, heartfelt concerns - and I always pay attention to that. So: no I am not burning through your patreon donations by travelling and moving. I am, in fact, using a combination of living quite frugally (just loving my new 4m x 4m flat) and pulling tricks like slipping in a conference visit for my 1-day-per-week employer into a country move (who says a business trip has to start and end at the same location?). All of this does take time, but: I will stay in Germany for quite a while now. And: most of this chaos is still somewhat due to realizing the hard way that Iceland is not a good place to live and GTFOing out of there earlier than planned.
Regarding what I did:
- The code of the activity network, applied to a tiny rural-village-scenario is starting to come together.
- I really tried to get something running as soon as possible, without spending a lot of time on implementing lots of details or making the architecture nice.
- I'm glad that I did, because it allows me to "fail fast", i.e. getting the chance to evaluate my assumptions as early as possible.
- My first focus is of course performance, to find out if my plans are feasible, or what I need to change in my approach to make them more feasible.
- Right now, it's still hard to tell how much too slow my naive first implementation is, but it is already very clear where the CPU has to spend too much work.
- One nice detail about my approach is that it is very introspectable, i.e. it is easy to look at what it is doing, what is going wrong, where which decision is taken.
That's all, I hope I'll have more to tell tomorrow!
I was travelling a lot last week and will travel some more this week, since I am moving countries again. No noteworthy progress yet, I'm still working on the setup for the rural-village-lifestyle-finding algorithm. I'll update you as soon as there is more!
This post is brought to you by cats.
So far, I'm still busy with preparation for the scary invent-a-magical-algorithm part of economy.
- I laid down the basic concepts of resources, places and activities in code
- I spent some time to come up with a good, compact data representation for activities, since the more activity-connections (of the activity network) I can fit into the CPU cache at once, any kind of search algorithm that I will write on top of them will run faster. Further optimization will probably make sense after I know the exact characteristics of the yet-to-be-fully-invented algorithm.
- Instead of just defining my initial (rural-town) resource, place and activity types directly in the code, I built a small system to load them from configuration files at runtime
- This will allow me to tweak individual values, or even introduce completely new resource, place and activity types without recompiling the game at all
- This means that some "pioneer modders" could already do the same and they don't even need developer tools, just a text editor!
- I'm actually trying to set a precedent here and see how much even complex behaviour can be treated as data and thus "outsourced" into configuration files - the more consequently I do this, the more accesible even powerful modding will be!
- My "data format" is actually just markdown with tables (specific column names and table cell formatting is expected from the code), which has the benefit of not only being easily editable, but looking extremely readable both in plaintext and when rendered on GitHub
- List of resources: plaintext, rendered
- Example of a place with activities: plaintext, rendered
- If you're wondering what the 'me', 'us', 'here' means: it refers to the transaction party being affected/referred to (me = current agent, us = current agent's family, here = household which offers this activity)
- I'm almost done with that
My next steps will be:
- Implement roads as activities (with dynamic costs/benefits)
- In code, generate a large (abstract, invisible) road grid with a random mix of homes, farms and shops at grid points, each with their activities
- Start sketching some working algorithm ideas for finding reasonable daily schedules in this synthetic-rural-town-grid
Let me know what you think!
This post is dedicated to Ian Paschal with whom I not only had very fruitful discussions about all kinds of things, but who, most importantly, keeps on telling me about the importance of game and interaction design, not as an afterthought, but as a main objective that is ideally explored and reached through iterative prototypes, which confirm something to be actually fun, or show that something is not right yet.
So dear Ian, I noticed your comments about this becoming very passionate (while still obviously being good critique and not naysaing). It seems as if my recent actions gave the impression that I only care about very detailed simulation and the technology behind that and that I will not pay enough attention to game design. I completely understand that it might look like that and I think it is my job to clarify some things that I think everyone will find interesting - hence the post.
First, what I'm doing right now is not even writing the game yet, it's more like writing the engine for the game. The Roads & Traffic prototype was a tech-demo, not a game prototype, and yes, the UI was atrocious and actually prevented many people from appreciating all that was in there. The economy prototype will also be mostly a tech-demo. Maybe I should communicate that better.
But: once I have these two ingredients (traffic + economy), interaction design will become the main focus for quite some time, and I want to do it in exactly the iterative way you suggest.
You are right that all this implies the assumption that this detail and sophstication will actually benefit gameplay - and this has been my central assumption indeed. Although with game design, as you say, words are pretty meaningless and the best theories don't help more than an interaction protoype, let me try to outline my motivation and explain how I want to make Citybound fun.
Let's assume that the context of a current-gen city building game is a given. Now let's see how Citybound will be different.
Starting with the planning mode, two things are interesting: first, my drawing tools, which I only showed as proof-of-concepts so far are already much more powerful and yet can be more intuitive than what other city builders have. Less frustration and more powerful expression should obviously be more fun. Second, the fact that you plan stuff and then implement the plans, instead of directly modifying the game world: I used to be worried that this obviously makes interacting with the game feel less direct and thus less fun. In reality, because you don't have to be so careful in planning mode since you can easily edit plans, you can actually experiment much more freely and it ends up feeling more direct - at least according to my own brief experience with the road and earlier planning tools.
Next, what I always lacked in city-building games was clear, understandable feedback beyond magically changing RCI demand bars and mysteriously rotting buildings. And this is exactly where detailed simulation actually starts having a purpose. For me it is very different to see that "there is demand for a school here (according to some statistics/number of people in the area)" versus "if you build a school here, these exact people will save this much time in their daily schedules, which could attribute for an average increase of their happiness of X, for example this family would now have enough time to go to a park together...". Once you get this level of detail not only in the simulation, but you expose it well, as feedback to player actions, I think you get very rewarding "I'm actually caring for real people here" choices for the player.
To bring the point home, I don't focus on the simulation and technology because it's academically interesting and cool in a nerdy way, instead of caring about game design - no, I actually do all of that to make completely new depths of gameplay possible, which will then of course need to be iterated to make sure it's not frustrating and actually achieves its potential. I hope with my reasoning I look not too ignorant while working on the technical underpinnings.
I've been noticing an increasing number of concerned people over my latest very ambitious and highly technical plans and relatively silent weeks that I spent on research. Let me address some of this in this post, paint a picture why I think I am doing what I need to be doing and tell you what will be next.
1. An apparent loss in momentum?
I am deeply aware of how silent I can be as a developer, especially after my "2016 year of silence and complete lack of visible progress".
I completely understand how anxious some of my decisions have made you, like:
- What, he will switch languages? Twice?
- What, he really wants to have millions of detailed cars?
- What, he really wants to simulate millions of citizens at an impossible-sounding degree of detail? (most recently)
I completely respect your feelings about this and I try my best to communicate my reasoning behind such things, but I understand that sometimes I will fail or under-explain things, and sometimes you will just never be able to appreciate why I'm doing things in a certain way, because you are not in my position.
So yes, sometimes it might look like not much is happening, like the project is doomed and so on, but in the meantime, I'm thinking or working silently on exactly the things that need to be done to make all of this great dream a possibility - and this is more important than always having steady visible progress - but of a much more mediocre result.
2. Yes, I want the "impossible"
Another concern I hear very often is "what you are trying to do is computationally infeasible", in varying degrees of technical sophistication and often with the conclusion: "why don't you just try something simpler", where something simpler is usually: crude approximations and extreme aggregation.
It's very easy to point out all the things that make something difficult, it is very hard to overcome the difficulties and it is crucial to even have the motivation to overcome them. I would love to share my exact motivation with you and thus maybe hear more encouraging voices:
Citybound will be all about interacting with the complex behaviours of city transportation and city economy that arise from the interactions and lifestyle choices of individuals.
Both SimCity and Cities: Skylines went some steps in this direction, but were to differing degrees severely limited by a) their technical underpinning and b) their much more humble goals.
I want a game where I can interact with a city of millions, where the fate of the city is a direct result of the fates of every single citizen. I want a game where I see the consequences of my city planning, not based on some statistical models or replacable generic agents, but based on the detailed lifestyle changes that I force onto unique citizens and the new opportunities that I open up for them.
If I succeed, we will not only have the richest, most infinitely interesting and fun-to-interact-with toy the genre ever had, we will have a playful tool to understand ourselves as city-dwellers in the context of millions of others and how our very own lifes are affected by policy, economy and processes which happen, almost imperceptibly, over years and decades. All of this would unfold in front of our eyes and in reaction to our actions in powerful and simple planning tools, so that even children can develop a visceral intuition about cities and large-scale human life that is severely needed to not just improve the real world, but first and foremost to understand and care.
At the same time, it would be the most all-encompassing and detailed model of city life that even research has seen (which is only starting to attempt complete simulations of some aspects of city life).
This all is the motivation I had since the beginning and it only has become clearer in its formulation over time.
3. And I will actually do it!
I find myself in a similar situation as a couple months ago, when I started writing the Roads & Traffic prototype. I did a lot of research on traffic simulation, didn't find something that suited my needs exactly, but had lots of inspiration and a rough model of how I wanted to implement it.
I wanted to show that with the new low-level architecture, I could actually achieve my goal of simulating ~500.000 cars with detailed behaviours. After a lot of optimization work, I had a working proof of concept that confirmed that it is possible.
Now, I want to do the same for the economy-simulation. Maybe the suprising and embarassing realization is that in all these years I didn't even have a good model for how I could achieve my goals of detail and realism in economy, but now I feel that I have a good model.
I did extensive and very thorough research of the best current work in this area, to make really sure I don't miss any existing ideas. Again, I didn't find anything that fits my needs exactly, but I got lots of inspiration along the way.
So again, starting to write actual code today, I will try my best to produce another proof of concept: that I can simulate millions of citizens at the level of detail I want and end up with a stable, reasonable economy.