Re: [vserver] Kernel oops with NULL pointer dereference when using cgroup hardlimiting

From: Herbert Poetzl <herbert_at_13thfloor.at>
Date: Sun 07 Feb 2010 - 17:58:54 GMT
Message-ID: <20100207175854.GG14282@MAIL.13thfloor.at>

On Sun, Feb 07, 2010 at 04:24:03PM +0100, Cryptronic wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1

> Hi,

> i have running a server with a 2.6.31.11-vs2.3.0.36.28 kernel.

> When using cgoups as descriped in
> http://linux-vserver.org/util-vserver:Cgroups#cgroup_and_CFS_based_CPU_hard_limiting_that_replaces_sched_hard
> after a view minutes i get a kernel null pointer:

could you test again with the current upstream kernel
and patch?
(http://vserver.13thfloor.at/Experimental/patch-2.6.31.12-vs2.3.0.36.28.diff)

and please enable DEBUG_INFO in your kernel config

TIA,
Herbert

> [ 300.166119] BUG: unable to handle kernel NULL pointer dereference
> at
> 0000000000000038
>
>
> [ 300.174083] IP: [<ffffffff8103668b>]
> set_next_entity+0xa/0x56
>
>
> [ 300.179918] PGD
> 0
>
>
> [ 300.182018] Oops: 0000 [#1]
> SMP
>
>
> [ 300.185383] last sysfs file:
> /sys/devices/virtual/net/lo/operstate
>
>
> [ 300.191590] CPU
> 14
>
>
> [ 300.193775] Modules linked in: ifb xt_tcpudp xt_state ipt_REJECT
> ipt_LOG xt_limit iptable_filter xt_DSCP xt_recent nf_nat_ftp nf_nat
> nf_conntrack_ftp nf_conntrack_irc nf_conntrack_ipv4 nf_conntrack
> nf_defrag_ipv4 ip]
> [ 300.252780] Pid: 65, comm: events/14 Not tainted
> 2.6.31.11-vs2.3.0.36.28-beng
> #1
>
>
> [ 300.261525] RIP: 0010:[<ffffffff8103668b>] [<ffffffff8103668b>]
> set_next_entity+0xa/0x56
>
>
> [ 300.269807] RSP: 0018:ffff881205e63d50 EFLAGS:
> 00010046
>
>
> [ 300.275160] RAX: 0000000000000000 RBX: ffff8811f5054680 RCX:
> 0000000000000000
>
>
> [ 300.282333] RDX: 0000000000000000 RSI: 0000000000000000 RDI:
> ffff8811f5054680
>
>
> [ 300.289506] RBP: 0000000000000000 R08: 0000000000000000 R09:
> 0000000000000000
>
>
> [ 300.296681] R10: 119026601aff243a R11: ffffc90001c10010 R12:
> ffff8811f5054680
>
>
> [ 300.303853] R13: ffffc90001c15708 R14: 0000000000000000 R15:
> ffffffff810d8248
>
>
> [ 300.311026] FS: 0000000000000000(0000) GS:ffffc90001c00000(0000)
> knlGS:0000000000000000
>
>
> [ 300.319166] CS: 0010 DS: 0018 ES: 0018 CR0:
> 000000008005003b
>
>
> [ 300.324952] CR2: 0000000000000038 CR3: 0000000001001000 CR4:
> 00000000000006e0
>
>
> [ 300.332126] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
> 0000000000000000
>
>
> [ 300.339300] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
> 0000000000000400
>
>
> [ 300.346472] Process events/14 (pid: 65, threadinfo
> ffff881205e62000, task
> ffff881205e61520)
>
>
> [ 300.354871]
> Stack:
>
>
> [ 300.356931] ffff8811f5054680 0000000000000000 ffffc90001c12d00
> ffffffff81037474
>
>
> [ 300.364347] <0> ffffffff812d29d0 ffffc90001c12d00 0000000000000000
> ffffffff81033138
>
>
> [ 300.372314] <0> ffffc90001c12d00 ffffc90001c12d00 000000000000000e
> ffffffff812c13d4
>
>
> [ 300.380521] Call
> Trace:
>
>
> [ 300.383020] [<ffffffff81037474>] ?
> pick_next_task_fair+0x85/0xa8
>
>
> [ 300.389156] [<ffffffff81033138>] ?
> pick_next_task+0x31/0x40
>
>
> [ 300.394861] [<ffffffff812c13d4>] ?
> schedule+0x58a/0x767
>
>
> [ 300.400218] [<ffffffff8104f5da>] ?
> lock_timer_base+0x26/0x4c
>
>
> [ 300.406006] [<ffffffff8104effa>] ?
> __round_jiffies_relative+0x12/0x17
>
>
> [ 300.412577] [<ffffffff810d8248>] ?
> cache_reap+0x0/0x20f
>
>
> [ 300.417931] [<ffffffff8105634a>] ?
> worker_thread+0xbb/0x20f
>
>
> [ 300.423634] [<ffffffff8105a456>] ?
> autoremove_wake_function+0x0/0x2e
>
>
> [ 300.430121] [<ffffffff8105628f>] ?
> worker_thread+0x0/0x20f
>
>
> [ 300.435735] [<ffffffff8105a10c>] ?
> kthread+0x8b/0x93
>
>
> [ 300.440831] [<ffffffff8100cb2a>] ?
> child_rip+0xa/0x20
>
>
> [ 300.446012] [<ffffffff8105a081>] ?
> kthread+0x0/0x93
>
>
> [ 300.451019] [<ffffffff8100cb20>] ? child_rip+0x0/0x20
> [ 300.456198] Code: 00 00 48 89 78 08 48 89 3a 48 89 93 e0 00 00 00
> 48 8b 53 08 48 8d 42 10 f0 80 4a 10 08 59 5e 5b c3 41 54 49 89 fc 55
> 48 89 f5 53 <83> 7e 38 00 74 24 48 8d 5e 10 48 39 5f 30 75 0d 48 89 df
> e8 d4
> [ 300.478538] RIP [<ffffffff8103668b>] set_next_entity+0xa/0x56
> [ 300.484463] RSP <ffff881205e63d50>
> [ 300.487997] CR2: 0000000000000038
> [ 300.491359] ---[ end trace 7151f959ecbc401e ]---
> [ 302.149450] BUG: unable to handle kernel NULL pointer dereference
> at 0000000000000038
> [ 302.157439] IP: [<ffffffff8103668b>] set_next_entity+0xa/0x56
> [ 302.163284] PGD 0
> [ 302.165400] Oops: 0000 [#2] SMP
> [ 302.168781] last sysfs file: /sys/devices/virtual/net/lo/operstate
> [ 302.174999] CPU 10
> [ 302.177202] Modules linked in: ifb xt_tcpudp xt_state ipt_REJECT
> ipt_LOG xt_limit iptable_filter xt_DSCP xt_recent nf_nat_ftp nf_nat
> nf_conntrack_ftp nf_conntrack_irc nf_conntrack_ipv4 nf_conntrack
> nf_defrag_ipv4 ip]
> [ 302.236280] Pid: 61, comm: events/10 Tainted: G D
> 2.6.31.11-vs2.3.0.36.28-beng #1
> [ 302.245805] RIP: 0010:[<ffffffff8103668b>] [<ffffffff8103668b>]
> set_next_entity+0xa/0x56
> [ 302.254084] RSP: 0018:ffff881205e57d50 EFLAGS: 00010046
> [ 302.259436] RAX: 0000000000000000 RBX: ffff8811f5054180 RCX:
> 0000000000000000
> [ 302.266609] RDX: 0000000000000000 RSI: 0000000000000000 RDI:
> ffff8811f5054180
> [ 302.273782] RBP: 0000000000000000 R08: 0000000000000000 R09:
> 0000000000000000
> [ 302.280954] R10: 11902660923a39f2 R11: ffffc90001410010 R12:
> ffff8811f5054180
> [ 302.288126] R13: ffffc90001415708 R14: ffff881205e554e0 R15:
> ffffffff810be9bc
> [ 302.295299] FS: 0000000000000000(0000) GS:ffffc90001400000(0000)
> knlGS:0000000000000000
> [ 302.303438] CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b
> [ 302.309227] CR2: 0000000000000038 CR3: 0000000001001000 CR4:
> 00000000000006e0
> [ 302.316402] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
> 0000000000000000
> [ 302.323575] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
> 0000000000000400
> [ 302.330749] Process events/10 (pid: 61, threadinfo
> ffff881205e56000, task ffff881205e554e0)
> [ 302.339148] Stack:
> [ 302.341208] ffff8811f5054180 0000000000000000 ffffc90001412d00
> ffffffff81037474
> [ 302.348626] <0> ffffc90001412d00 ffffc90001412d00 ffffc9000140ff70
> ffffffff8103312b
> [ 302.356594] <0> ffffc90001412d00 ffffc90001412d00 000000000000000a
> ffffffff812c13d4
> [ 302.364803] Call Trace:
> [ 302.367300] [<ffffffff81037474>] ? pick_next_task_fair+0x85/0xa8
> [ 302.373433] [<ffffffff8103312b>] ? pick_next_task+0x24/0x40
> [ 302.379134] [<ffffffff812c13d4>] ? schedule+0x58a/0x767
> [ 302.384490] [<ffffffff8104f5da>] ? lock_timer_base+0x26/0x4c
> [ 302.390279] [<ffffffff810be9bc>] ? vmstat_update+0x0/0x3a
> [ 302.395805] [<ffffffff810be9bc>] ? vmstat_update+0x0/0x3a
> [ 302.401332] [<ffffffff8105634a>] ? worker_thread+0xbb/0x20f
> [ 302.407033] [<ffffffff8105a456>] ? autoremove_wake_function+0x0/0x2e
> [ 302.413514] [<ffffffff8105628f>] ? worker_thread+0x0/0x20f
> [ 302.419128] [<ffffffff8105a10c>] ? kthread+0x8b/0x93
> [ 302.424223] [<ffffffff8100cb2a>] ? child_rip+0xa/0x20
> [ 302.429404] [<ffffffff8105a081>] ? kthread+0x0/0x93
> [ 302.434410] [<ffffffff8100cb20>] ? child_rip+0x0/0x20
> [ 302.439590] Code: 00 00 48 89 78 08 48 89 3a 48 89 93 e0 00 00 00
> 48 8b 53 08 48 8d 42 10 f0 80 4a 10 08 59 5e 5b c3 41 54 49 89 fc 55
> 48 89 f5 53 <83> 7e 38 00 74 24 48 8d 5e 10 48 39 5f 30 75 0d 48 89 df
> e8 d4
> [ 302.461949] RIP [<ffffffff8103668b>] set_next_entity+0xa/0x56
> [ 302.467875] RSP <ffff881205e57d50>
> [ 302.471408] CR2: 0000000000000038
> [ 302.474768] ---[ end trace 7151f959ecbc401f ]---
>
>
>
> Without cgroups feature all works well.
>
> vserver-info
> Versions:
> Kernel:vserver-info
> Versions:
> Kernel: 2.6.31.11-vs2.3.0.36.28-beng
> VS-API: 0x00020305
> util-vserver: 0.30.216-pre2864; Nov 29 2009, 00:39:14
>
> Features:
> CC: gcc, gcc (Debian 4.3.4-6) 4.3.4
> CXX: g++, g++ (Debian 4.3.4-6) 4.3.4
> CPPFLAGS: ''
> CFLAGS: '-Wall -g -O2 -std=c99 -Wall -pedantic -W
> - -funit-at-a-time'
> CXXFLAGS: '-g -O2 -ansi -Wall -pedantic -W
> - -fmessage-length=0 -funit-at-a-time'
> build/host: x86_64-pc-linux-gnu/x86_64-pc-linux-gnu
> Use dietlibc: yes
> Build C++ programs: yes
> Build C99 programs: yes
> Available APIs: v13,net,v21,v22,v23,netv2
> ext2fs Source: e2fsprogs
> syscall(2) invocation: alternative
> vserver(2) syscall#: 236/glibc
> crypto api: nss
> python bindings: no
> use library versioning: yes
>
> Paths:
> prefix: /usr
> sysconf-Directory: /etc
> cfg-Directory: /etc/vservers
> initrd-Directory: $(sysconfdir)/init.d
> pkgstate-Directory: /var/run/vservers
> vserver-Rootdir: /var/lib/vservers
>
> This also happens with 2.6.31.7-vs2.3.0.36.27. Maybe an upstream problem
> or vserver related?
>
> Best regards
>
> Oliver
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.14 (GNU/Linux)
>
> iEYEARECAAYFAktu2xMACgkQOBdlVlcPuhzQBQCgovcNHLlXkisBujQEpoiaZrIC
> ZPkAoMFbIapePBZUx1vJiyuAwlfJt+6u
> =WaNU
> -----END PGP SIGNATURE-----
Received on Sun Feb 7 18:01:46 2010

[Next/Previous Months] [Main vserver Project Homepage] [Howto Subscribe/Unsubscribe] [Paul Sladen's vserver stuff]
Generated on Sun 07 Feb 2010 - 18:01:50 GMT by hypermail 2.1.8