[Suggestion] Speed up the save process

[ApolloLV]

So, I don't know if this is a bug or, as I suppose, desired/expected behaviour, but Prison Architect freezes whenever I save the game. Since Autosave is enabled by standard, this occurs rather regularly, and lasts about 1,2 seconds, although this tends to become longer and longer with bigger maps and additional prisoners.
When I talked about this with a fellow CS student of mine, he suggested writing the save file to RAM first, then writing to disk. This way, the disk write time (which is still a major factor with the unfortunate of us that still have a HDD instead of a SSD) and spinup will not affect the game performance and the players won't even realize the game was saved (since writing to ram is incredibly fast).
The implementation of this would be pretty easy, I would suggest the method collecting the savegame information writing the info to a String, and after being done, invoking a method that writes the string to the disk while also releasing the game action, but I suppose the devs can come up with a simple, elegant solution.
And for those concerned with RAM usage, the .prison and .png files together are even on big maps rarely over 15 MB.

[5hifty]

I have Prison Architect running and Savin to a SSD, doesn't help.

[xander]

Writing to RAM first doesn't solve the problem, as the game is locking up (probably) because the process of saving is not shuffled off into another thread. If the save function is not handled by another thread, the game is going to have to wait for it to finish before moving on. Hence the delay. As implied above, I think that the correct solution is to off-load saving to a parallel thread.

xander

[Medu_Salem]

I'd also welcome if the way the games saves would be changed to be much faster or at least not as annoying as it is currently.

The bigger your map is or the more you build or the more prisoners you have the longer it takes to save... Sometimes it takes up to 20-30 seconds to save... on expanded large maps with 300-400 prisoners on an oldschool HDD. You can easily go to the toilet or go and grab a beer from the kitchen the time it takes to complete saving.

Literally EVERY time it's at least 20 seconds... every 10-15 minutes with autosave enabled.

I already had to adjust the auto-save time manually months ago in the preferences-file (didn't work from within the games menu) because I couldn't take it anymore that the game does this annoying freeze every 10 minutes.

[xander]

Y'all do know that you can change the autosave interval, right?

xander

[ApolloLV]

@Xander Yes, It is indeed obvious that no parallel processing takes places. But I assume this is deliberate to prevent data inconsistencies:
Just imagine an Object being handed over from one Person to another while the parallel saving takes place. When you load up that save later, the object will be carried by two persons at two places at once.
Sure, you could try to handle problems like this with priorities in the order items are saved, security mechanisms etc. But that's not a practical solution, especially with few devolopers working at IV.
So, assuming the current technique of stopping the whole game to save it is deliberate, I say we need to speed up saving itself. And while SSDs are pretty fast nowadays, RAM is our best bet for saving fast. And writing a save file that was consistently created from the RAM to whatever Disk you are using can then be parallelized and should have no effect on the core game performance at all.

Edit: Yes, obviously most people know about the autosave timer, but still, autosave has saved my ass pretty often, especially as we are dealing with a not-fully-stable beta here. And saving has to take place eventually!
I just don't want to feel like playing Sims, with save times that exceed 3 minutes. And if we are heading the right way, we may see even bigger and bigger prisons in the future, and a small problem (20 seconds is a PITA, but not severe) may become pretty serious.
After all, I think if I could get my hands on the source, it should not pose any problems at all, and IV has some WIZARDS working on it...

I would love to test how Disk speed affects the save process, but unfortunately, my Laptop is only equipped with a HDD.

Anyone with a dual HDD/SSD setup out there who would like to play GlaDOS and test a little? (No, a caching SSD won't do the job)

[xander]

...RAM is our best bet for saving fast.

You miss the point. Saving to RAM first saves no time because the game data still needs to be written to disk. Right now, the game stops to write to disk. Under your model, the game has to stop to write to RAM, then write to disk. Simply naively writing to RAM first saves no time, and actually makes the entire process of saving the game take *longer*. As I said the correct strategy is to parallelize the game saving process and knock it out into a different thread. This thread should take a snapshot of the game state, then write a save file to disk (which is exactly what the game is currently doing, though to take a snapshot of the current game state it simply freezes the game).

xander

[paktsardines]

The implementation of this would be pretty easy, I would suggest the method collecting the savegame information writing the info to a String

and

I say we need to speed up saving itself. And while SSDs are pretty fast nowadays, RAM is our best bet for saving fast.

I agree wholeheartedly! I cannot understate how much good sense these suggestions make! If you had a kickstarter I'd have already mortgaged my house.

In fact I have taken your suggestion and gone one step further and removed my HDDs and SDDs entirely, now the game saves really quickly in RAM!

Also, should I ever want to turn my computer off (what kind of a clown would ever do such a thing?), your suggestion to save the game as a string has proved invaluable. Now I just turn on my Canon, and print out the savegame on 12 A4 pages (only 6 or seven if I use A3).
Now I can safely turn my computer off and know I can resume my saved game any time I want, simply by scanning my 12 pages back into the game. I couldn't be happier!

edit: Oh, ninja'd. Hi xander.

[5hifty]

http://www.tomshardware.com/reviews/hyperos-dram-hard-drive-block,1186.html

[ApolloLV]

As it seems, I was not able to make my point to neither Xander nor Paktsardines.
Just to get my understanding of what the game is currently doing to the same point:
(snapshot is the current game state in text or binary in the RAM
Does it invoke a method that goes like
pause game -> make snapshot -> write snapshot to disk -> resume game ?
Or does it do
pause game -> write game state to disk -> resume game ?
My preposition is:
pause game -> make snapshot -> resume game
......................................... -> write snapshot to disk

Does any of you two know EXACTLY how the game is saved?
And BTW: sarcasm will get you nowhere productive...

[xander]

--==<snip>==--

We understand what you are proposing, but I think that you do not understand the problem. Whether the game writes the file to disk immediately or stores it in RAM for a while before writing to disk, it still has to freeze the game to write the data, since the process of writing the data hasn't been shuffled off into another thread. I suppose that it could write one character per game loop or something silly like that, but such a method would be an open invitation for data corruption---it is best to open a file for writing, write to the file, then release the file.

Again, the current problem is that the game *must* stop in order to write data to the disk and this can cause delays, especially with slow drives or busses. Nothing is gained by writing the file to RAM then writing it to disk. The only ways that I can think of to prevent saving from delaying the game for long periods of time are to compress the data before writing to disk (rendering the files slightly more difficult to edit, and still delaying the game, though for less time) or to remove game saving from the main thread and give it a thread of its own.

xander

[paktsardines]

Does any of you two know EXACTLY how the game is saved?

In all probability there is no need to create a 'snapshot' before writing to disk as every object/entity on the screen is already in memory, which serves as a 'snapshot' anyway.

So when saving, the game probably does 'pause game->write game state to disk->resume game'.

There are good arguments for 'pausing the game' while writing the save file.
Firstly, it takes time to write all the objects to disk and you don't want objects moving about in the interim. Also it allows more resources to be thrown at the save operation, meaning it is finished sooner.

Threading the save operation would be more trouble than it's worth. Somehow you'd need to create a 'snapshot' without pausing the game. That takes time and, as mentioned above, objects would be free to move about during that time. As it's threaded, that time is significantly extended (relatively) as full resources can't be thrown at the problem or game performance suffers. This means at time 0 (when the save starts) prisoner A's position might be saved but by the time it gets to saving prisoner Z's position, prisoner Z has moved. When you reload the game, objects won't be where you expect them, rather where they were some interval after saving.
The other downside is that if you solve the above, creating a 'snapshot' will instantly double your memory footprint. (You need to not only account for where all objects are in the 'playing' game, but where they all were when the save occurred).

edit: Sarcasm removed.

edit2: Thanks 5hifty... very interesting. Can you recommend a good A3 scanner?

[5hifty]

edit2: Thanks 5hifty... very interesting. Can you recommend a good A3 scanner?

Heh the argument was about saving the game to RAM first. I figured this would would do that, using the current save function

[paktsardines]

Sorry 5hifty, I realised that's what you were saying and was genuinely interested. The scanner joke obviously confused things.

[5hifty]

Ah ok. Yeah I wasn't sure which way the comment was leaning. They are a pretty crazy piece of technology ey? As long as the bottleneck was on the RAM used and not on the chassis that they sat in, you would be able to get some pretty crazy read/write speeds. Would suck if ya got a power out at some stage tho.... Unless you have some kinda UPS I'm pretty sure you would lose all ya data.