Let's say we have some specific nodes for specific tasks. E.g. we deployed Elasticsearch (3 nodes) and assigned Elasticsearch to those nodes with node-pool labels. We grew up and we need more resources for Elasticsearch node pool specifically. Currently, we have to:

Either:

1. Deploy new node-pool with a different name

2. Wait for deployment process

3. Change nodeSelector for Elasticsearch (if it's statefulset, it's even more complicated)

4. Wait until all pods move to different nodes

5. Kill old node pool

Or:

1. Destroy old node pool (downtime guaranteed)

2. Wait for node pool destruction

3. Create new node pool with same name, but different resources scale for nodes

4. Wait for everything to start up

Both processes are either complicated and require a lot of manual work, or have downtime.

It would be better if we could resize node pools by nodes sizes rather than node count only. For example, scale from 4 cores per node to 8 cores per node, with rolling node replacement rather than all-at-once.