Is there any documentation or guidelines on how to use pools? I've just been stuffing shapes etc into the same pool I created the simulation with. Is this fine/recommended/frownedupon/madness?
--michael
Pools
Re: Pools
Using the same pool for simulation and shapes in the simulation is fine, and probably best. Tracking which pool to use for disposing shapes could get annoying otherwise, and maximizing the use of one pool improves memory efficiency. (Pools preallocate blocks of memory, so if you have a bunch of pools that each have only a few allocations from them, most of the preallocated space is wasted.)
General recommendations would look like:
1) Use one pool unless there is a reason not to, but
2) having more pools than strictly necessary isn't really a problem. If it would be convenient to create a BufferPool for a side system that manages its own resource lifespans, that's totally fine.
3) BufferPools aren't thread safe.
4) When using thread local buffer pools (like the IThreadDispatcher's API), it can end up being convenient to treat all allocations as living only as long as the thread's execution. This can help avoid complex ownership tracking where it's no longer clear what pool owns what. It's also likely that the engine's own use of thread local memory pools through the IThreadDispatcher will change to region allocators eventually.
General recommendations would look like:
1) Use one pool unless there is a reason not to, but
2) having more pools than strictly necessary isn't really a problem. If it would be convenient to create a BufferPool for a side system that manages its own resource lifespans, that's totally fine.
3) BufferPools aren't thread safe.
4) When using thread local buffer pools (like the IThreadDispatcher's API), it can end up being convenient to treat all allocations as living only as long as the thread's execution. This can help avoid complex ownership tracking where it's no longer clear what pool owns what. It's also likely that the engine's own use of thread local memory pools through the IThreadDispatcher will change to region allocators eventually.
Re: Pools
Along these same lines, and related to the discussion here, I have questions about the pros and cons of these options:
1. Using a single static pool (or even a static collection of locked pools for multithreading)
1. Using a single static pool (or even a static collection of locked pools for multithreading)
- Reasoning: Being pools of memory, this is a system resource that you want tied to your threading model rather than your simulation model.
- Is there a downside to holding onto the same pools for the lifetime of the app? If we Return memory, but never Clear the pool, will it leak?
- Reasoning: Being memory associated with a simulation, you want the simulation and all shapes to coexist and be cleaned up together.
- Reasoning: Being independent sets of memory, you want the lifetime of a shape to be managed separately from the lifetime of a simulation so they can be used in a new simulation. This also makes it easier for content pipelines to load shapes without being handed a preexisting pool.
- Should we construct the pool with smaller minimumBlockAllocationSize and expectedPooledResourceCount if it will only be used for 1 or a few objects?
- You say in the linked thread that this is fine if you unpin or Clear. Clearing would clean up the memory immediately and we would only want to call this when we're done using the resource. What does setting Pinned to false do, and when would we do that -- when loading the resource or after we're done using it?
Re: Pools
A leak could happen if you let a BufferPool go out of use while it retains pinned references to memory. If you intend to let a BufferPool get garbage collected, Clear it first.Is there a downside to holding onto the same pools for the lifetime of the app? If we Return memory, but never Clear the pool, will it leak?
If you keep using a BufferPool for the lifetime of the application and let the OS process teardown do its thing without calling Clear, there will be no leak because process teardown doesn't much care about what the GC thinks. You might still get an assert from the BufferPool if the debug finalizer runs before teardown, but it's ignorable in that case since the OS is gobbling everything anyway.
Maybe- there's a bit of difficulty saying 'yes' because smaller allocations won't be put into the large object heap, but will still be pinned. That could interfere with the GC more than a LOH allocation that the GC probably won't move anyway.Should we construct the pool with smaller minimumBlockAllocationSize and expectedPooledResourceCount if it will only be used for 1 or a few objects?
If the lifespan of the pool (and its pins) is very short, the GC wouldn't be too bothered either way. Constantly recreating buffer pools with large backing allocations would probably hit some zero initialization costs, I suppose.
Setting Pinned to false allows the GC to move the memory referenced by the buffer pool, if it wants to. This means any outstanding buffers will become invalid, because the memory they pointed to might have moved.Clearing would clean up the memory immediately and we would only want to call this when we're done using the resource. What does setting Pinned to false do, and when would we do that -- when loading the resource or after we're done using it?
In the rare case where 1) there are no outstanding buffers and 2) you want to perform a GC and 3) you want the memory backing the buffer pool to be potentially compacted (perhaps because you're forcing a LOH compaction) and 4) you want to keep the previously allocated internal pools around, then setting Pinned to false for a little while makes sense. You can't take resources from the pool until it's Pinned again.
Unless you need to do exactly that, I would generally advise ignoring the existence of the Pinned property. It'll likely go away once the library moves off of .NET Standard 2.0 (in the .NET 5-or-later timespan).
On to general recommendations:
I'd say just do what's easiest and simplest. Avoid anything that involves locking multithreaded access, since that is effectively never the easiest. Having a pool associated with a worker thread that it uses for ephemeral allocations, regardless of what the worker thread is working on, is one easy way to handle things.
If possible and convenient, avoid the complexity of tracking which buffer pool a given allocation came from. If you're creating a bunch of Mesh shapes from different pools, you'll need to know which buffer pool allocated what when they get disposed. You could store metadata for that, but it's a bit of a pain.
In many cases, persistent allocations can be preallocated by one central pool without locking, and then worker threads can use those allocations. This is often the simplest approach when it's possible. Ephemeral allocations could still be done on per-thread pools without adding complexity since the allocation will get returned to the thread pool before the worker finishes.
Re: Pools
Thanks! I'll have to take more time to think this through, but I guess the takeaways I'm getting are
- Ignore Pinned.
- Don't mess with the constructor default params unless you know what you're doing.
- Taking and Returning from the same pool over a long period is fine, as long as you don't let it get GC-ed before Clearing.
- Using a collection of pools is annoying because you have to worry about contention, and also track which allocation came from which pool when you Return.
- Worker threads with ephemeral allocations can use a per-thread pool as long as they're returned before the worker finishes.
- Persistent allocations of known sizes or upper limits can be preallocated in a central pool and passed to worker threads.
- Easy: use a single pool and don't use worker threads at all
- Hard: use a collection of pools (either one per worker or locking carefully for thread-safe access), and track which allocation came from which pool
Last edited by jnoyola on Thu Jul 09, 2020 1:32 am, edited 1 time in total.
Re: Pools
Pretty much, with one small note: using a collection of pools does not necessarily imply locking and contention provided that every worker has a guarantee it's the only one touching a pool; the annoyance is primarily ownership tracking.
Also, assuming that the shapes you're concerned about are primarily meshes, it's worth noting that there's a Serialize function and a constructor that takes serialized data. It'll avoid the majority of the cost of creating a mesh.
I'd be a little leery about aggressively clearing BufferPools all the time, since their whole point is to give the GC a break. It likely won't be a visible concern either way at load time, but if you find it easy and convenient to reuse a BufferPool without clearing it and you have the memory available, it would probably be faster since it doesn't need to get the GC involved as often.
Also, assuming that the shapes you're concerned about are primarily meshes, it's worth noting that there's a Serialize function and a constructor that takes serialized data. It'll avoid the majority of the cost of creating a mesh.
Clearing unpins and drops all references to backing memory, letting the GC consume all of it. That would be sufficient to free up memory back to baseline and the BufferPool can be reused. Of course, no real harm in creating a new one and letting the old cleared BufferPool get eaten too if it's convenient.If your pool grows to using 100MB RAM, does Clearing reset that back to the baseline? Or do you have to GC that pool and create a new one? For example when you're unloading the previous level and loading a new one (or even a menu, which may have far less memory requirements), is it enough to Clear the pool, or should you construct a new pool?
I'd be a little leery about aggressively clearing BufferPools all the time, since their whole point is to give the GC a break. It likely won't be a visible concern either way at load time, but if you find it easy and convenient to reuse a BufferPool without clearing it and you have the memory available, it would probably be faster since it doesn't need to get the GC involved as often.
Re: Pools
Good point. Updated to note that.using a collection of pools does not necessarily imply locking and contention provided that every worker has a guarantee it's the only one touching a pool
My thought was more for convenience of not having to swap out the BufferPool reference if I can instead just Clear it between levels. But I suppose I can just as easily swap it out with a new one as long as all systems access it from the source instead of holding onto references.but if you find it easy and convenient to reuse a BufferPool without clearing it and you have the memory available