Download your Coursera lectures using JS, bash and wget

So, in light of recent events, I was unable to follow up on my Coursera Algorithms course.

Trying to find a way to download all the lectures at once, I couldn’t find a way that worked for me – I found two, but they simply didn’t work. With the help of some JavaScript that I got from one of these sources, I cooked up the following steps to easily download all the lectures at once:

Step 1: Extract the lecture names, and their IDs from the Coursera page
Navigate to your lecture page(for me, it was https://class.coursera.org/algs4partII-003/lecture).
Execute the following JavaScript in the Console(in Chrome, use the Inspect Element option):

function findLectures () {
  var link_elems = document.querySelectorAll('.lecture-link');
  var lectures = "";
  Array.prototype.slice.call(link_elems).forEach(function (elem, i) {
    var lecture = i + "|" + elem.getAttribute('data-lecture-id') + "|" + elem.innerText.trim();
    lectures += lecture + "\n";
  });
  return lectures;
}

Then call the method findLectures(). This will print each lecture in the following format: Lecture position in page|Lecture ID|Lecture name.
Example output:

0|43|Course Introduction (9:22)
1|1|Introduction to Graphs (9:32)
2|2|Graph API (14:47)

Save this to a file named lectures.txt(exclude any quotes that get pasted)

Step 2: Export Coursera’s cookies
Use the Chrome plugin cookie.txt export to export all the cookies that Coursera has saved. Save the result to a file named cookies.txt

Step 3: Download them!
Run the following in the terminal:

export COUNT=1; while read line; do lecture_name=`echo $line | awk -F '|' {'print $3'}` lecture_id=`echo $line | awk -F '|' {'print $2'}`; echo "Downloading lecture $COUNT; ID=$lecture_id; Name=$lecture_name..."; ((COUNT++)); wget --load-cookies cookies.txt https://class.coursera.org/algs4partII-003/lecture/download.mp4?lecture_id=$lecture_id -O "$COUNT - $lecture_name"; done < lectures.txt

Make sure you replace the link to where your lectures are listed. The above link has “https://class.coursera.org/algs4partII-003/lecture/” in it.

This will download all the lectures in the current working directory. It does not maintain any relationship with the week that the lecture was released in. It downloads them in the order that they are listed on the Coursera lecture page.

The Unofficial Gentoo Linux x86 uClibC stage3s’

Update:
The catalyst spec files are available here:
https://github.com/judepereira/gentoo-development/tree/master/x86/uclibc/catalyst

Following this Gentoo bug:
https://bugs.gentoo.org/show_bug.cgi?id=441976
We have official experimental uClibC for x86 and amd64 stages.
I will not be maintaining these unofficial ones any longer.

The link to the official archives is:
http://mirrors.rit.edu/gentoo/experimental/x86/uclibc/
and
http://mirrors.rit.edu/gentoo/experimental/amd64/uclibc/

The uclibc experimental stages on the gentoo mirrors are all outdated(they go back to years), so here are my stage3s’, which have been updated. They’re very similar to, and in fact can be considered to be the same stages that the Gentoo Community provides, only updated.

As of 27th August 2011, they are now built by catalyst.

The stages are built by catalyst now, and I’m looking for the possibility of hosting them on the Gentoo mirrors.

Follow my tutorial here, to get a basic ideology on working with it

stage3-uclibc-x86-26062012 is affected by bug #423491 sys-apps/findutils-4.4.2-r1 and sys-apps/coreutils-8.14 with sys-libs/uclibc – rpmatch.c:58:1: error: redefinition of ‘rpmatch’ /// /usr/include/stdlib.h:810:28: note: previous definition of ‘rpmatch’ was here

The workaround for it right now is to comment the entire function in stdlib.h and those packages will compile well. Also comment the _wur definition, just above the function rpmatch. This is a crude workaround, and it’ll be fixed when upstream decides what’s the solution.

Files:
26th June, 2012
stage3-uclibc-x86-26062012.tar.bz2
stage3-uclibc-x86-26062012.tar.bz2.md5sum

The above stage was not build by catalyst, as it could not deal with the rpmatch bug. This stage was created manually from the last release.

5th August, 2011
stage3-uclibc-x86-02082011.tar.bz2
stage3-uclibc-x86-02082011.tar.bz2.md5sum

7th July, 2011
stage3-uclibc-x86-07072011.tar.bz2
stage3-uclibc-x86-07072011.tar.bz2.md5

1st June, 2011
stage3-uclibc-x86-01062011.tar.bz2
stage3-uclibc-x86-01062011.tar.bz2.MD5

25th April, 2011
stage3-uclibc-x86-25042011.tar.bz2
stage3-uclibc-x86-25042011.tar.bz2.md5sum

For historic releases, post a comment, as to which one you require and why a specific version.
If you want the stage1 or stage2, drop a comment and I shall have them uploaded.

For those of you who would like to build these stages themselves, here are the spec files for catalyst:
stage1.spec
stage2.spec
stage3.spec

Update:
The catalyst spec files are available here:
https://github.com/judepereira/gentoo-development/tree/master/x86/uclibc/catalyst

In addition, I’ve taken down all the stage files that I used to host here, in favour of the official stages now available from the Gentoo mirrors:

Use Ruby to Generate your Shadow Password

I was initially stumbled on creating the shadow compatible SHA-512 hash.
After a little research, the answer was obvious:

require 'digest/sha2'

password = "pass@123"
salt = rand(36**8).to_s(36)
shadow_hash = password.crypt("$6$" + salt)

And you now have a password hash which you can directly use in /etc/shadow

recursive get from an FTP server

As we all know, downloading a directory from an FTP server over FTP over the command line is not possible. Some, have found mget to be the holy grail, I certainly do not.

I was restoring my music backup from my NAS, and I didn’t want to do it the GUI way, so CLI is the other, obvious option.

Trying to do a recursive fetch using the FTP prompt, did not work whatsoever.

Here is the better solution: recursive wget.



$ wget -r -l 0 ftp://user:password@server/directory

wget will now recursively fetch that entire directory.

Note: In some cases, you would need to pass the –username and –password arguments to wget.

Resetting a User’s Home Permissions

Recently I managed to mess up my home permissions, to the extent that I owned the files, but couldn’t read them.

Fixing an entire user’s home directory permissions is simple and easy.

# chown user:user -R /home/user/
# chmod -R u+rwX /home/user/

Done.

comparison sheet | plug computers

A lot of these plug computers are in the market. Which one is right for you? Here’s a comparison sheet to help you decide which one should you be looking for:

Plug Computer Processor Memory JTAG WiFi Network USB SATA Price
Ionics Cumulus KW 1.2Ghz 512MB NAND, 512MB DDR2 Yes Yes 1 Ethernet 1 2
Seagate Dockstar KW 1.2Ghz 256MB NAND, 128MB RAM Make One No 1 Ethernet 3 0 $30
Pogo Plug Pro No Yes 1 Ethernet 4 0 $99
Open RD Client Sheeva 1.2Ghz 512MB NAND, 512MB DDR2 Yes No 2 Ports 7 2 $250
Beagle Board TI 600Mhz 256MB NAND, 128MB RAM Yes No 1 Ethernet 1 0 $150
Guru Plug KW 1.2Ghz Yes No 2 Ports 2 1 $99
Guru Plug Server KW 1.2Ghz 512MB NAND, 512MB DDR2 Yes 2 Ports 3 1 $129
Marvell Sheeva Plug KW 1.2Ghz 512MB NAND, 512MB DDR2 Yes No 1 Ethernet 1 0 $99

If anyone can fill in the missing values, please leave a comment

playing with grsecurity | a brief tutorial

This howto is intended for those looking for better means to secure the Linux kernel, and the userland by the means of a powerful and simple role based access control policy.

Contents

  1. What is grsecurity?
  2. Setting up grsecurity + gentoo in a VM
  3. The real thing: Grsecurity
    1. Getting familiar with gradm
    2. Generating the policy
    3. Fixing the errors
    4. Roles, subjects and objects
    5. The include directive
    6. Best Practices for /etc/grsec/policy
  4. Filtering grsecurity logs with rsyslog

What is grsecurity?

grsecurity is an innovative approach to security utilizing a multi-layered detection, prevention, and containment model. It is a set of patches for the Linux kernel with an emphasis on enhancing security. Its typical application is in web servers and systems that accept remote connections from untrusted locations, such as systems offering shell access to its users.

Extensive information about grsecurity can be found from the following links:

This tutorial briefly gives you an introduction on using grsecurity. The grsecurity wikibook is written by the creator(Bradley Spengler) of the subject in discussion.

Setting up grsecurity + gentoo in a VM

To test grsecurity’s features, we’ll setup gentoo hardened in a virtual machine using QEMU. Let’s get QEMU installed.

  • Redhat: yum install qemu qemu-img
  • Debian: aptitude install kvm
  • Gentoo: emerge -av qemu-kvm with the following use flags: aio hardened jpeg ncurses png qemu_softmmu_targets_arm qemu_softmmu_targets_i386 qemu_softmmu_targets_x86_64 qemu_user_targets_x86_64 ssl

Create a raw image and set it up:

# qemu-img create -f raw grsec.gentoo.img 2G
# fdisk grsec.gentoo.img
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0xe2c8c9c9.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4, default 1): 1
First sector (2048-4194303, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-4194303, default 4194303):
Using default value 4194303

Command (m for help): a
Partition number (1-4): 1

Command (m for help): p

Disk grsec.gentoo.img: 2147 MB, 2147483648 bytes
255 heads, 63 sectors/track, 261 cylinders, total 4194304 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xe2c8c9c9

	   Device Boot      Start         End      Blocks   Id  System
grsec.gentoo.img1   *        2048     4194303     2096128   83  Linux

Command (m for help): w
The partition table has been altered!

Syncing disks.
# losetup -v -o $((512*2048)) -f grsec.gentoo.img
Loop device is /dev/loop0
# mkfs.ext4 -L "grsec.gentoo" /dev/loop0
# mkdir grsec.gentoo-rootfs
# mount /dev/loop0 grsec.gentoo-rootfs

Now download the hardened gentoo stage3. Follow the chapters 5 to 10, of the handbook keeping in mind the following set of instructions (gentoo x86_64 handbook):

  • Select the profile “hardened/linux/amd64”
  • For networking, choose the DHCP method. We’ll discuss how to create custom rules for ssh.
  • When doing anything related to GRUB, see below
  • Install the kernel source: =sys-kernel/hardened-sources-2.6.38 and =sys-apps/gradm-2.2.2.201103262019 . If necessary, unmask them.
  • The extra packages that I installed are dhcp openssh eix pciutils vim gentoolkit rsyslog vixie-cron grub =sys-kernel/hardened-sources-2.6.38 =sys-apps/gradm-2.2.2.201103262019
  • Kernel config for the QEMU envionment is listed as follows.

Kernel configuration for the environment(only essentials):

-> Processor type and features
  -> Processor family
    -> Core 2/newer Xeon (set this to your host CPU)

-> Device Drivers
   -> Serial ATA and Parallel ATA drivers
     -> ATA SFF support
       -> ATA BMDMA support
         -> Intel ESB, ICH, PIIX3, PIIX4 PATA/SATA support

-> Device Drivers
  -> Network device support
    -> Ethernet (10 or 100Mbit)
      -> RealTek RTL-8129/8130/8139 PCI Fast Ethernet Adapter support

-> File systems
  -> The Extended 4 (ext4) filesystem
    -> Use ext4 for ext2/ext3 file systems

-> Security options
  -> Grsecurity
    -> Grsecurity
      -> Security Level
        -> Hardened Gentoo [virtualization]
    -> Filesystem Protections
      -> Restrict /proc to user only
  -> Restrict unprivileged access to the kernel syslog

That’s all that’s essential, go ahead and drop extra things that just make the kernel fat. Make sure you disable Paravirtualized guest support under Processor type and features in any case. Enabling that causes the system to not boot.

GRUB menu entry:

title gentoo hardened
root (hd0,0)
kernel /boot/vmlinuz-2.6.38-hardened root=/dev/sda1

Exit the chroot. Unmount the filesystem, and prepare it for booting. DO NOT unmount anything yet.

# losetup -v -f grsec.gentoo.img
Loop device is /dev/loop1
# echo "(hd0) /dev/loop1" > /tmp/device.map
# grub --device-map=/dev/null
    GNU GRUB  version 0.97  (640K lower / 9216K upper memory)

     [ Minimal BASH-like line editing is supported.  For the first word, TAB
       lists possible command completions.  Anywhere else TAB lists the possible
       completions of a device/filename. ]

grub> device (hd0) /dev/loop1

grub> root (hd0,0)
Filesystem type is ext2fs, partition type 0x83

grub> setup --stage2=/path/to/grsec.gentoo-rootfs/boot/grub/stage2
 Checking if "/boot/grub/stage1" exists... yes
 Checking if "/boot/grub/stage2" exists... yes
 Checking if "/boot/grub/e2fs_stage1_5" exists... yes
 Running "embed /boot/grub/e2fs_stage1_5 (hd0)"...  18 sectors are embedded.
 succeeded
 Running "install --stage2=/path/to/grsec.gentoo-rootfs/boot/grub/stage2 /boot/grub/stage1 (hd0)
  (hd0)1+18 p (hd0,0)/boot/grub/stage2 /boot/grub/menu.lst"... succeeded
 Done.

grub> quit

Now startup the virtual machine.

$ qemu-kvm -cpu qemu64 -smp 2 -m 384M -hda grsec.gentoo.img -curses \
-net user -net nic,model=rtl8139 -redir tcp:8022::22

This should bring up an ncurses based UI to the VM.

The real thing: Grsecurity

If you’ve noticed from the kernel configuration, you’ve set restrict /proc to user only. Create a user unauth login to it. top will show you only the processes owned by that user. ls /proc will confirm that. Further dmeg will tell you Operation not permitted. These are just a few basic security enhancements. You can go ahead and set the custom profile in the kernel config and setup the system as you like. For now, we’ll discuss the RBAC system of grsecurity.

Getting familiar with the RBAC system of grsecurity: gradm

gradm is a tool to enable, disable, and control the RBAC system of grsecurity. It is a very powerful tool. Let’s start from the beginning, setting passwords, and enabling the full learning process.
Use gradm -P to set the master password. This is used to disable and reload the RBAC mechanism. gradm -P to set the password of any user role defined in the policy file.

# gradm -P
Setting up grsecurity RBAC password
Password:
Re-enter Password:
Password written to /etc/grsec/pw.
# gradm -P admin
Setting up password for role admin
Password:
Re-enter Password:
Password written to /etc/grsec/pw.
# gradm -P shutdown
Setting up password for role shutdown
Password:
Re-enter Password:
Password written to /etc/grsec/pw.

Full system learning:

grsecurity has a feature called learning. In the beginning you can enable the full learning process, where grlearn will log all your actions. Any actions that you do not want other users access to, must be done by logging into the RBAC system. To enable full system learning, we do the following, create a new file /etc/local.d/gradm.start with the following contents:

/sbin/gradm -FL /grlearn.log

Make sure you make the file executable, by chmod +x /etc/local.d/gradm.start
On your next boot, it will automatically start. Add the service sshd and rsyslog to start by default and reboot:

# rc-update add sshd default
# rc-upadte add rsyslog boot
# reboot

Now you can ssh into the machine, by:

$ ssh root@localhost -p 8022

Do something, and then check the logs, ssh into the VM from another terminal and tail -f /grlearn.log
You’ll see many entries when you so some specific resource actions. Example, uptime, wget. Lots of entries are generated. System packages need only to be installed under the role of admin. So login to role admin, and install a package. Restarting the sshd service is a good idea too.

# gradm -a admin
Password:
# emerge -avuDN vim

Notice that nothing is logged in the grlearn.log file. To logout of that role, simply do a gradm -u

Generating the policy

Once the full learning process is done, or rather you’ve done everything that you don’t want access to in the user role admin, its time to generate the policy. For desktop users, you may want to run the full learning process for a day or two. First disable it, then generate the policy and append it to the default policy.

# gradm -FL /grlearn.log -O /etc/grsec/flearn.policy
Beginning full learning 1st pass...done.
Beginning full learning role reduction...done.
Beginning full learning 2nd pass...done.
Beginning full learning subject reduction for user sshd...done.
Beginning full learning subject reduction for user root...done.
Beginning full learning object reduction for subject /...done.
Beginning full learning object reduction for subject /etc/init.d...done.
Beginning full learning object reduction for subject /sbin/rc...done.
Beginning full learning object reduction for subject /usr/bin/wget...done.
Beginning full learning object reduction for subject /usr/sbin/sshd...done.
Full learning complete.
# cat /etc/grsec/flearn.policy >> /etc/grsec/policy

Fixing the errors:

If you simply do gradm -E, it wouldn’t work. Let’s look at the errors and fix them

# gradm -E
Duplicate object found for "/lib64" in role shutdown, subject /, on line 257 of /etc/grsec/policy.
"/lib64" references the same object as the following object(s):
/lib (due to symlinking/hardlinking)
/lib64 (due to symlinking/hardlinking)
specified on an earlier line.  The RBAC system will not load until this error is fixed.

Open the policy file with your favorite editor, and go to that line. Then comment that line out, as its already protected because it’s a symlink to somewhere else. Check the policy file for errors again, do gradm -C. Another error, great! Yes, just comment out all those lines, they’re the same reference. Then at last you’ll come to the good error:

# gradm -C
Duplicate role admin on line 463 of /etc/grsec/policy.
The RBAC system will not be allowed to be enabled until this error is fixed.

This duplicate is because the default policy which is put in place by installing gradm, contains an admin role already. So go to those lines and comment them.

#role admin sA
#subject / rvka
#       / rwcdmlxi

If you see an error like the following, then you have to add the variable $grsec_denied to that particular role.

# gradm -C
Viewing access is allowed by role root to /proc/kcore [...]

Reading access is allowed by role root to /proc/slabinfo [...]

Reading access is allowed by role root to /proc/modules [...]

Reading access is allowed by role root to /proc/kallsyms [...]

There were 4 holes found in your RBAC configuration.  These must be fixed before the RBAC system will b
e allowed to be enabled.

I’ll explain roles, subjects, objects and modes later further down in this tutorial. For now, search for the string “role root” in the policy file, and follow downwards from there, untill you come to -CAP_ALL, add $grsec_denied just before that.
Now the grsecurity RBAC system is ready to be loaded, once gradm -C reports no errors with the policy file. Enable it then, and see if it’s working

# gradm -E

Now in the syslog you should see this:
2011-05-21T05:11:15.536086+00:00 localhost kernel: [ 3070.796712] grsec: From 10.0.2.2: (root:U:/sbin/gradm) grsecurity 2.2.2 RBAC system loaded by /sbin/gradm[gradm:1767] uid/euid:0/0 gid/egid:0/0, parent /bin/bash[bash:1542] uid/euid:0/0 gid/egid:0/0

# ls /etc/grsec
ls: cannot access /etc/grsec: No such file or directory

Viola! The RBAC system is running flawlessly!
Now if you ssh into the system, it will fail. Take a look at the logs(/var/log/messages):

2011-05-24T03:15:08.836286+00:00 localhost kernel: [ 2011.183591] grsec: From 10.0.2.2: (root:U:/) denied open of /proc/1677/oom_score_adj for writing by /usr/sbin/sshd[sshd:1677] uid/euid:0/0 gid/egid:0/0, parent /usr/sbin/sshd[sshd:1527] uid/euid:0/0 gid/egid:0/0
2011-05-24T03:15:08.836320+00:00 localhost kernel: [ 2011.183822] grsec: From 10.0.2.2: (root:U:/) denied access to hidden file /usr/sbin/sshd by /usr/sbin/sshd[sshd:1677] uid/euid:0/0 gid/egid:0/0, parent /usr/sbin/sshd[sshd:1527] uid/euid:0/0 gid/egid:0/0
2011-05-24T03:15:08.837263+00:00 localhost kernel: [ 2011.184485] grsec: From 10.0.2.2: (root:U:/) denied connect() to the unix domain socket /dev/log by /usr/sbin/sshd[sshd:1677] uid/euid:0/0 gid/egid:0/0, parent /usr/sbin/sshd[sshd:1527] uid/euid:0/0 gid/egid:0/0
2011-05-24T03:15:08.857339+00:00 localhost kernel: [ 2011.204857] grsec: From 10.0.2.2: (root:U:/) denied connect() to the unix domain socket /dev/log by /usr/sbin/sshd[sshd:1677] uid/euid:0/0 gid/egid:0/0, parent /usr/sbin/sshd[sshd:1527] uid/euid:0/0 gid/egid:0/0
2011-05-24T03:15:08.860352+00:00 localhost kernel: [ 2011.207771] grsec: From 10.0.2.2: (root:U:/) use of CAP_SYS_CHROOT denied for /usr/sbin/sshd[sshd:1678] uid/euid:0/0 gid/egid:0/0, parent /usr/sbin/sshd[sshd:1677] uid/euid:0/0 gid/egid:0/0

The text in bold indicates that that particular user is denying access to that particular resource. Login to the admin role(gradm -a admin), and search for the string “role root”, then “subject /usr/sbin/sshd”. Add the following entries in the right places(stick to the alphabetical order, its easier to read):

        /proc/*/oom_score_adj      rw
        /usr/sbin/sshd                  rx
        /var/empty                      r

Reload the RBAC system, you should be able to ssh now. There are still a few errors in the logs. You can simply add the two following lines to fix them.

        /var/log/tallylog           rw
        /proc/*/loginuid           rw

This should fix everything that ssh needs access to.

Roles, subjects and objects

The structure takes the following pattern:

role <user> u
<user attributes>

subject <app/binary> o
    <object>                <mode>

subject <app/binary> <mode(s)>
    <object>                <mode>

The letter ‘u’ follows the username mentioned to indicate that it is a user. ‘g’ can be used for groups. Attributes include transitions, ip_allow, etc. A complete detail of this is found on the grsecurity wikibook.

Subjects have members called objects to define what access does the subject in question have. There are various modes here too. The ‘o’ in the first subject is to make sure inheritance is not followed from the default policy or user policy.

The include directive

Once your familiar with the policy file, reinstall the default one, and use the include directive in the main policy file to include a certain directory containing policies for different users/subjects etc.

A reasonable structure would be to have:

/etc/grsec
/etc/grsec/policy.d
/etc/grsec/policy.d/user1
/etc/grsec/policy.d/user1/logs
/etc/grsec/policy.d/user1/policy

To learn the actions of a new user, you can have the following in the main policy file(at the end of it):

role jude ul

Then start the learn process with gradm -L /etc/grsec/policy.d/jude/learning.logs -E. Generate the policy from the logs, and place the policy as basic.policy in the policy directory. Once your done with that, delete the line you’ve added for the learn and replace it with:

include </etc/grsec/policy.d/user1/policy>

 

Best Practices for /etc/grsec/policy

  • Make the policy as restrictive as possible, there are many features that haven’t been touched in this tutorial, read the grsecurity wikibook for a complete hands on grsecurity.
  • Keep things simple, keep to the alphabetic order.
  • Create a policy and directory structure the way your comfortable with.
  • In the beginning of the policy file, there are lot of details about the present RBAC system, read them.

Filtering grsecurity logs with rsyslog

If your using rsyslog, you may want to filter out the grsecurity messages. Append the following lines to /etc/rsyslog.conf, and restart rsyslog

# grsec logs
:msg, contains, "grsec" /var/log/grsecurity.log
:msg, contains, "grsec" ~

If you encounter any problems whilst following this tutorial, feel free to comment

the one thing bing.com is useful for : backgrounds

714 wonderful backgrounds!

Yes, in my personal opinion, bing.com is only useful in supplying fresh backgrounds, which are breath taking. These breath taking set of wallpapers, when set to change randomly on your desktop, makes the look and feel more exquisite.

Download the file(see below). The script and the images are inside the tar file. Untar them in your home directory, chmod +x bing_change_wallpaper.sh and place the file in /home/user/bin/ directory. Rename the images folder to .bing_wallpapers and place it in your home. The script by default changes the wallpaper every thirty minutes, you can edit the script to your satisfaction. For GNOME only, click on System -> Preferences -> Startup Applications, with the following entries.

Name: bing backgrounds switcher
Command: /home/user/bin/bing_change_wallpaper.sh
Comment: bing backgrounds switcher


Files:

[howto] grsecurity + NOUVEAU + Compiz + Seg Fault

Assuming that you have a grsec + PaX enabled kernel, you would realise that the nvidia-drivers are a bad choice. Quite a few applications will fail(the ones that use libGLcore.so). Use the nouveau driver for your card, as it’s pretty much stable and works with good 3D acceleration.

compiz under NOUVEAU + PaX

To get compiz working NOUVEAU under hardened linux, first enable the kernel DRM module for nouveau. Follow this link: The X Server Configuration HOWTO

Build the kernel, and install it. Edit the VIDEO_CARDS variable in your make.conf to say only nouveau, nothing more, nothing less.

Unmask the following packages: media-libs/mesa, x11-libs/pixman, x11-drivers/xf86-video-nouveau, x11-base/xorg-drivers, x11-base/xorg-server, x11-libs/libdrm, x11-drivers/xf86-input-evdev, x11-drivers/xf86-input-keyboard, x11-drivers/xf86-input-mouse

Install the above packages, make sure you’ve done a emerge -C nvidia-drivers nvidia-settings prior to the merge.

Reboot the system, it should all work out of the box, compiz will fail with a segmentation fault, look into your logs. You’ll see something like the following:

2011-05-11T17:22:24.760922+05:08 halcyon-82 kernel: [ 2026.893377] grsec: denied
 resource overstep by requesting 4096 for RLIMIT_CORE against limit 0 for /usr/b
in/compiz[compiz:20146] uid/euid:1000/1000 gid/egid:1000/1000, parent /bin/bash[
bash:11847] uid/euid:1000/1000 gid/egid:1000/1000

2011-05-11T17:26:07.848848+05:08 halcyon-82 kernel: [ 2249.981362] compiz[20378]
: segfault at ffffffffffffffff ip 00000284c5f39fa1 sp 0000039c50e0ee00 error 6 i
n nouveau_dri.so[284c5cc3000+38b000]

Simply disable pax for compiz and emerald, do the following as root:

# paxctl -zm /usr/bin/compiz
# paxctl -zm /usr/bin/emerald

Now, start compiz as usual and your all set.
On a side note, flash player will show a similar issue too, so disable PaX for that too.

WARNING: Disabling PaX for compiz, emerald and flash is a security risk.

creating unique environments – chroot snapshots – using aufs2

What if you decided that you wanted to experiment and test within a chroot(ed) environment? And then something went wrong and you had to start all over again from scratch? Big headache, too much pain, especially with RPM based distributions.

Well don’t do that then, here’s an interesting approach. Use aufs2, a stackable filesystem. An actual filesystem with “RAID-like” support.

chroot layers, using aufs2

You can inherit different environmental settings from various bases, much like the concept of object oriented programming.

Aufs is a stackable unification filesystem such as Unionfs, which unifies several directories and provides a merged single directory.(from http://aufs.sourceforge.net/)

Setup aufs2. You need to patch your kernel. For gentoo, you can just merge sys-fs/aufs2 with the following flags: “fuse inotify kernel-patch kernel_linux ramfs -debug -hardened -hfs -nfs”. The patch will be automatically applied, you just need to recompile it after merging. For other distributions, take a look at the homepage of aufs2.

Setup your chroot as you normally do. Let’s say you’ve setup centos in a chroot environment, for developing. You’ve done yum groupinsall “Development Libraries” “Development Tools”. Let’s create a snapshot here of this environment.

# mkdir env1
# mount -t aufs -o br=/home/jude/env1/:/home/jude/centos-dev=ro none wor
k-dir
# chroot work-dir

 
What does this do? As you can see, your first centos-dev has been branched as RO, read-only. Which means, any changes now made, are transparently written into env1 directory, with the very same structuring. Say, you’ve structured env1 as you want it to be, now you need a second environment, based on the first env1, you can simple do:

# mkdir env2
# mount -t aufs -o br=/home/jude/env2/:/home/jude/env1/=ro:/home/jude/ce
ntos-dev=ro none work-dir
# chroot work-dir

 
Now, env1 is RO, any changes now are further written to env2, keeping the structuring of every little change intact, just as the first. If you want to suddenly use env1, don’t add env2 to the RW branch, any branch that doesn’t have anything in front of it, is assumed to be RW, read-write.

What happens when you delete a file/directory from a branch mounted as RO? A file called .wh.filename is created in the RW branch, indicating to the overlaying filesystem to NOT index that file anymore.

In this way, you could create several different environments, each being inheriting a base, and then depending on the needs, the other environments created.

You can comment if you get any errors, I’d be glad to help you out.

bootstrap CentOS from Gentoo (or any linux distribution)

To bootstrap CentOS in Gentoo, I did the following. It basically installs the essential CentOS components into the specified directory, and from there on, chroot into that directory and perform tests, etc. First off, you’ll need to unmask a few packages(replace ~amd64 with ~x86 if your on the x86 architecture), and enable the flag sqlite for rpm. Execute the following commands as root:

# echo sys-apps/yum ~amd64 >> /etc/portage/package.keywords
# echo dev-python/sqlitecachec ~amd64 >> /etc/portage/package.keywords
# echo app-arch/rpm sqlite >> /etc/portage/package.use

 
Now we’re ready to install the primary packages required to bootstrap CentOS. Merge the following atoms: sys-apps/yum app-arch/rpm dev-python/m2crypto as

# emerge -avuDN sys-apps/yum app-arch/rpm dev-python/m2crypto

 
In this example, we’ll bootstrap CentOS 5(x86_64). Download the stage2(sounds like gentoo now, huh?) from any CentOS mirror. The minimal stage2 does not work with this process.

# wget http://mirrors.seas.harvard.edu/centos/5.6/os/x86_64/images/stage2.img

 
You need to extract the contents of this squashfs image file, so make sure you have =sys-fs/squashfs-tools-3.1 installed. The latest one in portage is broken. Execute the following:

$ unsquashfs-3.1 -d /devel/centos5 stage2.img

 
Now we have a working CentOS 5.6 installation. Install bash and yum into it before you can finally chroot.

# yum --installroot /devel/centos5/ --nogpgcheck install bash
# yum --installroot /devel/centos5/ --nogpgcheck install yum

 
The –nogpgcheck is so that yum doesn’t check the gpg keys which are currently missing on your host gentoo system, or any other non-rpm distribution. chroot into /devel/centos5 and perform some cleaning:

# chroot /devel/centos5
# mkdir /old
# mv /var/lib/rpm /old
# yum install yum

 
What did you just do? Well the RPM DB generated by your host distribution’s RPM, will certainly not match CentOS’s yum, hence we remove the stale RPM database. Everything in the stage2 comprises only of the dependencies of yum and yum itself. Doing a yum install yum will fix everything.
 
Congratulations, you now have the base CentOS installation in /devel/centos5 These instructions are not limited to Gentoo alone, but to any distribution. Any questions may be posted as comments, and I’ll be glad to help you out.