TL; DR:

• Don't optimize before necessary
• Spend time designing upfront - it'll save you time in the long run (writing/thinking is faster than coding)

Background:

Christmas Village Creator started without an idea - I simply wanted to make Christmas village pixel art but wasn't so sure what the gameplay would be. Through the process, I found that I was having so much fun mocking up villages and placing the different assets, I started thinking that other people would have fun doing the same thing. Which led me to make the "game" a sandbox where you can place assets around and create your own village.

Half the fun of creating something is sharing it, so I decided I'd look into hosting player data and allowing it to be shared between players. I had just used SilentWolf Godot backend services for a leaderboard in a previous game jam and I knew they offered player data services as well, so I decided to give it a try. SilentWolf itself has been great - it's free, easy to integrate into your Godot project, actively developed, and only requires a few lines of code (check out the code for my project here). Plus, I've had great luck with getting responses from SIlentWolf support.

The problems that I ran into had nothing to do with SilentWolf services - my issues were of my own creation; a result of decisions about the structure of the data.  I used two different data formats - one for storage and one for in-game use at runtime - which resulted in more work than there needed to be transforming the data back-and-forth (not the best thing to burden yourself with on a time-limited game jam).

Problem:

When a village is first loaded in the game, all the village data is fetched upfront from SilentWolf. To update the data with a player's edits, all the latest village data is sent to SilentWolf when the player opens the menu and presses 'Save' (I decided to not attempt any "real-time updates" given the amount of traffic to SilentWolf that might incur and the complexity of implementing incremental updates performantly).

The format of the data sent & retrieved from SilentWolf is structured like this:

// Data storage format, simplified (omits house data)
{     
    "id": "<village_id>",     
    "name": "Murphy's Dad's Village",     
    "village": {         
        "objects": {             
            "<object_type_1>": [                 
                {                     
                    "id": "<object_id_1>",                     
                    "x": 0.0,                     
                    "y": 0.0                 
                },                 
                {                     
                    "id": "<object_id_2>",                     
                    "x": 1.0,                     
                    "y": 1.0                 
                }             
            ]         
        }     
    } 
}

My primary concern when deciding on this format was the size of the data. My concern was that any single village could possibly hold hundreds or thousands of objects and I wanted to be conscientious about my storage footprint in SilentWolf. So I decided on this format, where I could avoid including the '<object_type_1>' ID with each data object and I estimated that I could reduce the total storage size by almost 25%!

However, the above structure didn't match the needs at runtime. I needed real-time updates to a local copy of the data because the object placement needs to be cached when the player enters or exits a house in the village. The above structure of the data I stored in SilentWolf wouldn't be performant enough for object deletions (when the player removes an object from the village). With the above structure, consider the case where the village map has ~1000 trees and they're all stored in a list - a deletion operation might need to iterate through all 1000 trees to find the tree that needs to be removed from the data (O(n) time-complexity).

So after fetching the data, I transformed it to this format (and translated it back when saving):

// Runtime data format
{
    "objects": {
        "<object_id_1>": {
            "type": "<object_type_1>",
            "x": 0.0,
            "y": 0.0
        },
        "<object_id_2>": {             
            "type": "<object_type_1>",            
            "x": 1.0,             
            "y": 1.0         
        }
    }
}

Having the data in this format at runtime makes the '<object_id_1>' IDs keys in dictionaries, which enables performant object deletions because the keys can be directly looked up for deletion (O(1) time-complexity).

Doing all this data transformation back-and-forth was what created a lot of work for me on the project. I didn't think it would take that long to implement when I first decided to transform the data, but it ended up taking several hours of work (critical time in a game jam).

Retrospect:

In retrospect, I think I was overoptimizing before necessary. I was concerned about burdening SilentWolf with an exorbitant amount of data, but I should've taken the approach of doing the minimum to get the thing done, then optimizing. Storage optimizations weren't needed at this "stage in the game".

Instead, I should've either:

a) Only used the second data format, both for data storage and at runtime. The data optimizations weren't needed yet, but the runtime optimizations are.

b) As I was writing this devlog, I realized a third data format could've fulfilled both of my needs (both runtime and storage optimization):

{
    "objects": {
        "<object_type_1>": {
            "<object_id_1>": {
                "x": 0.0,
                "y": 0.0
            },
            "<object_id_2>": {                 
                "x": 1.0,
                "y": 1.0             
            }
        }
    }
}

I don't know why I didn't think of this when I developed the game. I think it was because I was moving so fast trying to get it done, I didn't really stop and take the time to think and design. That's why you take the time to think and design before jumping into the implementation!

Conclusion:

• Don't optimize before necessary
• Spend time designing upfront - it'll save you time in the long run (writing/thinking is faster than coding)