Search This Blog

Friday, May 29, 2015

SWAP! Problem and Solutions


SWAP! Problem and Solutions

Swap such a lovely and a very well known word in the Linux world. Something everyone who has ever used Linux in any form should be aware of.

Swap is an area of the Hard Disk which is used as a temporary storage of files by the Linux kernel. While using any program on a computer, the binaries of the program must be loaded into the system memory/RAM so that the processor can execute the code. This is a standard and almost all Operating systems were designed to work like this. Linux is no exception. Linux Kernel as an inbuilt algorithm that calculates the code that could be safely put to the Swap area thereby saving some RAM.

The history of computers tells us that the most valuable system resource for a computer for executing any program after the CPU is the RAM. And history also tells us that this very resource has always been scarce.RAM was very scarce and expensive hence this affected many important design choices for Operating Systems. The very architecture of Operating Systems especially Linux was hence put forth with an Idea to conserve as much RAM as possible for more immediate tasks, and use RAM only for code in constant use by programs.Use it less and try using an alternative resource for code which gets loaded into the Memory but is not used all that frequently. So a separate partition was created the SWAP partition.

The Swap partition was traditionally that specific area of the Hard Disk which was allocated while installing a Linux System for Swapping purpose. Swapping is essentially temporarily moving less used code from a running program to a Hard Disk area. This reduced pressure on the RAM resource and also ensured that the system was able to run and do its job. Linux as an OS always has respected RAM by design, Linux Memory Management is the best because of that. Linux tends to put little Pressure on RAM and only load up code in RAM which is constantly being used by the running programs. It quickly moves the less frequently used code on to the Swap are.

Well Swapping is fine and indeed something that should happen when you are using Linux on a headless server. All you use to connect to the server is a ssh connection. The Server is most likely serving web pages or sending emails. Using headless server means you are making the best use of the System Resources. The Operating System takes very less resources to run while almost all system resources are allocated for doing the more Important tasks like serving your web pages or transferring emails. Such systems have no user interface. The end user is not directly interacting with the server and all the user cares about is the web pages and emails they use. So technically swapping is not going to slow down your system. It will in fact help in case the system load increases beyond what you expected. This is one of the reasons why Linux is so successful in Servers and Mainframes.

The Scenario on a Desktop on the other hand is completely different that this. For Starters, a user is constantly interacting with the desktop. Applications are launched and closed. A complete Graphical subsystem is run on a Desktop. This is something that is not typical of a Linux Use case, However since Linux is free and Open Source, there are people who are passionate about making Linux Desktop a success. Consequently there is a large number of projects tackling this in their own way. Hence we have a choice of different desktop environments.

So where does Swap Stand in this. Although we have so many Desktop Environment and distributions, the Linux Kernel is still the Same as it was years ago. Support for new hardware is added all the time so the Experience on the Desktop has gotten better over the years. SWAP however remains almost the same concept. Swap is still swap and the way Linux Kernel uses swap is still the same. Linux will still try and conserve RAM by using swap. This is the way Linux is designed. This is fine for Server Use Case. In case of the Server use, Swap is a boon and a lifesaver at times. The users using the Services from the server never see any Impact. They never feel that the Server is swapping. Using Swap this is a very normal phenomena in Server Space. With the Desktop however, the software that runs is different the way system is used is different. So If we go by they way Linux uses swap, there may be certain parts of the code loaded into the memory that the system may not access that frequently but they are needed to be in RAM for an acceptable experience. Linux kernel is agnostic to this an carries on with the memory management algorithms it was designed with. The swapping on a Desktop can create more issues than it solves. Swapping makes the Desktop Environment and the application graphics less responsive. This potentially degrades the user experience. System feels a lot slow.

Luckily though we can control the degree of swappiness, We can control the severity of swapping. We can tell the Linux kernel to either not use swap at all or use swap when a certain threshold is reached on the RAM consumption. We could tweak the percentage after which swapping to disk should happen. This at least guarantees some level of performance and you know exactly when your system would start swapping out to Disk. This make things a lot better, however we are still not near the Ideal solution.

This is a well known fact that Swapping could kill your desktop experience however there are certain advantages which you definitely get with having a swap partition on the disk. Suspend/resume and Hibernation work really well with Swap on disk. This is a feature which a lot of users using Linux on Desktop appreciate.

While suspend and resume are important to people however I personally feel that the downside of excessive swapping is way more than the advantages of being able to swap and hibernate to the disk. I personally don't use swap on my Laptop/Desktop unless I have an SSD where the Swap resides. Swapping to an SSD can considerably reduce the lag that could result from swapping to a spinning disk.

So there must be a solution to this problem. We all know that the Linux Kernel needs a swap area, That is how the Kernel was designed. So we needed to device a solution which would provide swap space and still would not kill the performance and responsiveness of the desktop. We indeed achieved that with recent versions of Linux Kernel. We now have support for ZSWAP and ZRAM in the Linux kernel. However these are still not enabled by default. Based on your distribution of choice and whether or not you are running from a SSD, you could tweak these to achieve best results.

ZSWAP :- This technology basically compresses the SWAP pages in memory before writing to the Disk. Consequently the CPU is taxed a bit. However instead of writing say 1 GB to the swap area which is essentially the same disk used traditionally, It would now write say 500MB. Which means it is using less swap. This could be very helpful in improving the system response on the desktop. Since the swap is on disk, you can still enjoy Suspend/Resume/Hibernation features of Linux on the Desktop.

ZRAM:- Zram is similar to Zswap except it does not use Disk at all to swap. It uses a portion of RAM instead and stores Swap pages there in compressed format. Since there is no disk involved, Everything is running out of RAM. This means the performance is going to be the best with this solution. It really works well. The compression is done well and Kernel is still able to swap but RAM is used instead. This will tax your CPU a bit because of the compression it does, however the experience is just fantastic. One caveat though, Since Disk is not used at all, You can't Suspend/Resume/Hibernate. That is one disadvantage you need to live with, This however gives you the best possible performance even though your system uses Swap.

On top of Zswap and Zram, One must tweak vm.swappiness to ensure that swapping is the last thing that happens on your Desktop. Linux Kernel must wait till maximum amount of RAM is consumed before starting to swap. So once RAM is scarce, Swapping happens however it either happens to RAM in a compressed format or it happens to an already allocated block of RAM.

I woud personally set vm.swappiness=0 or at the most vm.swappiness=10
Hope you liked this and you can now choose between ZRAM and ZWAP and not use just swap!!

Monday, May 4, 2015

How to Install the latest Nvidia Driver in Ubuntu 14.04.2/14.10/15/04 via Bumblebee

How to Install the latest  Nvidia Driver in Ubuntu 14.04.2/14.10/15/04 via Bumblebee    

This is a followup post to one of my earlier posts which was appreciated by the community and I felt like I was able to help people with my work. I would like to carry on with this. I know for a fact that the only problem with installing Ubuntu on a Laptop with Optimus Nvidia Graphic Cards is that people get black screen after installing the Proprietary Nvidia Drivers. I would like to step up and be a source of help for Getting bumblebee installed and configured on these laptops so people could continue using Ubuntu.

OK first things first. If you want to understand how Bumblebee Works? How it handles the Hybrid Graphics? Please read this first.


This is my earlier blog post which provides some clarity on what Bumblebee is and how it works

Now lets get down to business. I am going to focus on getting the Latest Nvidia Driver 349.16 on Ubuntu 14.04 and later.

The official Repository of Ubuntu has only old and dated drivers. I don't use that myself and would suggest you do the same if you want the latest Nvidia Driver without any problems.

Steps:-

1) First step after installing Ubuntu is to install all updates. You may use the software updater or simply run sudo apt-get update && sudo apt-get upgrade

2) Add the Xorg-edgers ppa using

sudo apt-add-repository ppa:xorg-edgers/ppa and then sudo apt-get update

Update :- There is just one more PPA which you need to add to get the Nvidia-Drivers now.

ppa:graphics-drivers/ppa, Please add this PPA as well only then will you be able to install latest Nvidia Drivers.
 
3) Run sudo apt-get upgrade && sudo apt-get dist-upgrade

This is the most Important step. If you miss this you will see a black screen after reboot. Please do this as we are using the xorg-edgers ppa. We need to ensure all packages required for bumblebee and nvidia are taken from the ppa.

4) Now Install bumblebee and Nvidia drivers using

sudo apt-get install bumblebee bumblebee-nvidia primus nvidia-349

5) Add the current user to the bumblebee security group.

sudo gpasswd -a $USER bumblebee

for Ubuntu 15.04 only you need to manually enable to bumblebee daemon using

sudo systemctl enable bumblebeed

6) This will take care of the Installing the required software on your system. We need to configure it.

I can assure you if you followed these steps so far then configuring should be a breeze.

We start by enabling bbswitch and intel i915 Driver.

7) sudo gedit /etc/modules 

Ubuntu 15.04 the file is /etc/modules-load.d/modules.conf

add these two lines to it

i915
bbswitch

8) Edit the bumblebee configuration file.
sudo gedit /etc/bumblebee/bumblebee.conf

1. line 22:

Driver=nvidia

2. line 55:

KernelDriver=nvidia-349

3. line 58:

LibraryPath=/usr/lib/nvidia-349:/usr/lib32/nvidia-349

4. line 61:

XorgModulePath=/usr/lib/nvidia-349/xorg,/usr/lib/xorg/modules

In addition to this there is another file that needs to edited.

9) sudo gedit /etc/modprobe.d/bumblebee.conf

Add the below lines to end.

#349
blacklist nvidia-349

Now you may reboot. After reboot.

Check using

sudo apt-get install mesa-utils if mesa-utils is not already installed.

primusrun glxinfo | grep OpenGL

This should tell you that nvidia-349.16 driver in use.
Please note that nvidia-349 is the latest driver at the time of writng this. Please check the launchpad page for xorg-edgers ppa an accordingly replace nvidia-349 with the latest version if any.

Let me know if this works for you.