Skip to content

GCS: periodically update cgroup limits#2827

Open
ryankeithster wants to merge 1 commit into
microsoft:mainfrom
ryankeithster:cgroup-memory-resize
Open

GCS: periodically update cgroup limits#2827
ryankeithster wants to merge 1 commit into
microsoft:mainfrom
ryankeithster:cgroup-memory-resize

Conversation

@ryankeithster

Copy link
Copy Markdown
Contributor

Summary

Currently, pod cgroup memory limits are set once during guest boot, and then never changed, not even in the case of a memory resize event. This can result is containers being OOM killed even when they have not come close to exceeding the available memory. These changes add a task that runs every 60 seconds in the GCS that polls for memory changes and then adjusts the pod cgroup memory limit accordingly.

Changes

Add periodic cgroup resize task to (cmd/gcs/main.go)

  • Task runs every 60 seconds and adjusts the pod cgroup memory limit if the gcs host system total memory has changed.

New test coverage (cmd/gcs/main_test.go)

Signed-off-by: Ryan Keith (from Dev Box) <ryankeith@microsoft.com>
@ryankeithster
ryankeithster requested a review from a team as a code owner July 17, 2026 20:30
@rawahars

Copy link
Copy Markdown
Contributor

This can have a couple of side-effects-

  • Say an update request updated the pod which exceeded the available limit. We might return successfully but later this new code path will fail.
  • There is a maximum gap of 60 seconds which is error prone.

Instead of this, can we simply perform the update of pod cgroup as part of the actual Update request?

Also, perhaps the Update request targeting pod-id should be explicitly used by caller to update the pod limit. I feel that GCS should not perform any implicit operation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants