[howto] grsecurity + NOUVEAU + Compiz + Seg Fault

Assuming that you have a grsec + PaX enabled kernel, you would realise that the nvidia-drivers are a bad choice. Quite a few applications will fail(the ones that use libGLcore.so). Use the nouveau driver for your card, as it’s pretty much stable and works with good 3D acceleration.

compiz under NOUVEAU + PaX

To get compiz working NOUVEAU under hardened linux, first enable the kernel DRM module for nouveau. Follow this link: The X Server Configuration HOWTO

Build the kernel, and install it. Edit the VIDEO_CARDS variable in your make.conf to say only nouveau, nothing more, nothing less.

Unmask the following packages: media-libs/mesa, x11-libs/pixman, x11-drivers/xf86-video-nouveau, x11-base/xorg-drivers, x11-base/xorg-server, x11-libs/libdrm, x11-drivers/xf86-input-evdev, x11-drivers/xf86-input-keyboard, x11-drivers/xf86-input-mouse

Install the above packages, make sure you’ve done a emerge -C nvidia-drivers nvidia-settings prior to the merge.

Reboot the system, it should all work out of the box, compiz will fail with a segmentation fault, look into your logs. You’ll see something like the following:

2011-05-11T17:22:24.760922+05:08 halcyon-82 kernel: [ 2026.893377] grsec: denied
 resource overstep by requesting 4096 for RLIMIT_CORE against limit 0 for /usr/b
in/compiz[compiz:20146] uid/euid:1000/1000 gid/egid:1000/1000, parent /bin/bash[
bash:11847] uid/euid:1000/1000 gid/egid:1000/1000

2011-05-11T17:26:07.848848+05:08 halcyon-82 kernel: [ 2249.981362] compiz[20378]
: segfault at ffffffffffffffff ip 00000284c5f39fa1 sp 0000039c50e0ee00 error 6 i
n nouveau_dri.so[284c5cc3000+38b000]

Simply disable pax for compiz and emerald, do the following as root:

# paxctl -zm /usr/bin/compiz
# paxctl -zm /usr/bin/emerald

Now, start compiz as usual and your all set.
On a side note, flash player will show a similar issue too, so disable PaX for that too.

WARNING: Disabling PaX for compiz, emerald and flash is a security risk.


3 responses to “[howto] grsecurity + NOUVEAU + Compiz + Seg Fault”

  1. peter harmsen Avatar
    peter harmsen

    Did you use the hardened profile?
    If so what options during kernel config did you enable/disable?
    What USE flags are necessary for a simple desktop for watching video and listening to music ,read some pdf’s?

    1. Yes,
      I used the hardened profile. In my kernel config, I had set the grsec configuration to the workstation profile. I’ve enabled almost all of the security features, and it worked well.

      As for the USE flags, just merge gnome, and then look for codecs in the USE flags for all the dependencies. Just add them as global USE flags, and it’ll work well.

      However, if your new to Linux, I suggest you get your hands on Fedora or something similar.

  2. nouveau seems to be more easy on the system, compiz takes up lesser amount of RAM, as of what I’ve observed.