I figured this out with Bertl's help on IRC.
Here's how to configure hard-cpu limits on Vserver devel branch to get
all of your guests to share the CPU equally:
1) This requires the use of idle-time scheduling, which is supported
in the devel-branch. Also, it turns out there was a subtle bug with
idle-time scheduling that was fixed with a recent patch. So, you need
the following kernel patches:
patch-2.6.17.7-vs2.1.1-rc28.n.diff
delta-sched-fix01.diff
2) When compiling the kernel be sure to enable the following vserver options:
CONFIG_VSERVER_HARDCPU=y
CONFIG_VSERVER_IDLETIME=y
CONFIG_VSERVER_IDLELIMIT=y
3) The vserver-util userspace tools do not yet support configuring
idle-time. So, you need to download and build the following tools:
http://vserver.13thfloor.at/Experimental/TOOLS/vsched-0.03.tar.bz2 (required)
http://vserver.13thfloor.at/Experimental/TOOLS/vcmd-0.03.tar.bz2
(optional -- can be used in place of vsched and for testing that idle
time scheduling works)
http://vserver.13thfloor.at/Experimental/TOOLS/cpuhog.c (optional
-- for testing that idle time scheduling works)
4) Build vsched, vcmd, and cpuhog on the vserver host.
5) We will set the regular hard cpu rate/interval to an unrealistic
value such as 1/1000 for all of the vserver guests. This will cause
the guests to halt and their processes to go idle. We will configure a
seperate "idle" bucket with idle_rate/idle_interval values of
something reasonable like 50/100. With the idletime and idlelimit
flags set in kernel config, instead of running the idle task, vserver
will advance idle time and add tokens to the guests according to
idle_rate/idle_interval. Of course, this assumes that the host server
is idle, too. This will cause the guests to self-regulate their cpu
usage and to share equally the cpu. This works in SMP systems, too.
Note: if you have a busy host, idle time may not advance as expected.
6) Start some guests and have them run cpuhog (we use vcmd for testing here):
vcmd -i 42 -BC ctx_create .flagword=^34^8 -- cpuhog &
vcmd -i 43 -BC ctx_create .flagword=^34^8 -- cpuhog &
vcmd -i 44 -BC ctx_create .flagword=^34^8 -- cpuhog &
8.) Run vsched to configure the cpu limits. We need to run it twice:
once to configure regular rate/interval, set the idle time flag, and
set the min and max bucket size; and another time to configure
idle_rate/idle_interval:
vsched -f -x <vserver_context_id> -u -1 -I 1000 -R 1 -M 50 -S 500 -i
vsched -f -i -x <vserver_context_id> -u -1 -I 100 -R 50
the '-f' option overrides lazy-update and updates the cpu limit
settings even if the guest is on hold.
Also, note that the vserver needs to be running for vsched-0.03 to
work. You can use a post-start script to automatically call
vsched-0.03 whenever you start the guest. You cannot yet use
/etc/vservers/<vserver-name>/schedule to set the idle-time flag,
idle_rate, or idle_interval.
9) Watch idle time advance and monitor the scheduler to confirm that
everything is working:
watch -n 4 /proc/virtual/<xid>/sched
You want to pay attention to the cpu<N> line(s). Here's what the
values on each cpu<N> line represent:
<user-ticks>
<sys-ticks>
<hold-ticks>
<token-time> in ticks
<idle-time> in ticks
<H/R> on hold/running
<I/-> idle time/normal
<tokens>
<tokens-min>
<tokens-max>
<fillrate>/<interval>
<fillrate2>/<interval2>
If <idle-time> is not advancing and your guests are on hold, run top
and confirm that your host really is idle (x% id).
-ekc
On 8/9/06, EKC <webmaster@generalsynthesis.com> wrote:
> Hello,
>
> I am trying to figure out how to use the idle time feature in the
> devel branch and the vsched-0.02 tool.
>
> What I want to do is have P guests each use 1/P of host CPU time and
> to dynamically adjust to use 1/(P+Y) of CPU time when Y new guests are
> started. When guests are stopped, I want hard_sched to adjust
> accordingly for the remaining guests.
>
> I am using hard_sched now and I have to re-run vsched for every guest
> every time that I start a new guest. I understand (thank you Bertl for
> enlightening me on IRC) that the idle-time feature in the devel branch
> will take care of this for me.
>
> What's a simple example of how to use vsched0.02
> (http://vserver.13thfloor.at/Experimental/TOOLS/vsched-0.02.tar.bz) to
> have P guests dynamically adjust to use 1/P of cpu time? Is the new
> vsched the best tool for this job?
>
> What would I need to do differently if I have more than 1 CPU?
>
> Thanks a lot!
>
> ekc
>
_______________________________________________
Vserver mailing list
Vserver@list.linux-vserver.org
http://list.linux-vserver.org/mailman/listinfo/vserver
Received on Tue Aug 15 04:22:07 2006