Linux 2.6.23 -- a backyard adventure in graphic driver hacking

How to get vesafb-tng and Nvidia's X driver 9755 working with the new kernel

Craig Goodrich

October 2007

Recent notes


OK, so you download 2.6.23 and are all set to build it, adding the last vesafb-tng patch first, of course. But patch frighteningly can't find the very first file and asks you the dreaded question "File to patch?"

Well, it's OK, just skip that particular patch and the rest goes in fine. Make xconfig and build as usual.

Now, of course, you have to build the Nvidia X driver module. Your old faithful NVIDIA-Linux-x86-1.0-97xx fails to build, so you go get the latest 100-something from It installs and builds just fine, X runs, but the machine seems locked up when you leave X -- or ctrl-alt-Fn to switch to a console. In fact, only the console screen hasn't been restored correctly and is blacked out; everything is fine but you can't see it. BUG!!!

(I should add here: if this happens to you, do not just hit the reset switch. Exit from X normally, count slowly to about five to let Gnome, KDE, or whatever do its housekeeping, and type "reboot" -- you'll be typing blind, of course. This way everything goes down in an orderly fashion.)

Sigh... Back to kernel 2.6.22? Take heart! There is hope!

Get the newest old driver (9755) from Nvidia. Extract it ( -x ); there will be a tree NVIDIA-Linux-x86-1.0-9755-pkg1/usr/src/nv/ full of source.

Now apply this patch:

--- a/NVIDIA-Linux-x86-1.0-9755-pkg1/usr/src/nv/nv-linux.h
+++ b/NVIDIA-Linux-x86-1.0-9755-pkg1/usr/src/nv/nv-linux.h
@@ -524,12 +524,12 @@
     {                                                \
         free_pages(ptr, order);                      \
+// only 5 args in kernel 2.6.23
 #define NV_KMEM_CACHE_CREATE(kmem_cache, name, type)            \
     {                                                           \
         kmem_cache = kmem_cache_create(name, sizeof(type),      \
-                        0, 0, NULL, NULL);                      \
-    } 
+                        0, 0, NULL);                            \
+    }
 #define NV_KMEM_CACHE_DESTROY(kmem_cache)                       \
     {                                                           \

--- a/NVIDIA-Linux-x86-1.0-9755-pkg1/usr/src/nv/nv.c
+++ b/NVIDIA-Linux-x86-1.0-9755-pkg1/usr/src/nv/nv.c
@@ -107,7 +107,7 @@
 static int nv_mmconfig_failure_detected = 0;
-static kmem_cache_t *nv_pte_t_cache = NULL;
+static struct kmem_cache *nv_pte_t_cache = NULL;	//	kernel 2.6.23
 // allow an easy way to convert all debug printfs related to events
 // back and forth between 'info' and 'errors'
@@ -1551,8 +1551,7 @@
         if (apm_nv_dev[i] != NULL) pm_unregister(apm_nv_dev[i]);
-    if (unregister_chrdev(nv_major, "nvidia") < 0)
-        nv_printf(NV_DBG_ERRORS, "NVRM: unregister nv chrdev failed\n");
+    unregister_chrdev(nv_major, "nvidia") ; // kernel 2.6.23 -- this now returns void (don't ask why)
     for (i = 0; i < num_nv_devices; i++)
@@ -1578,9 +1577,8 @@
     nv_printf(NV_DBG_INFO, "NVRM: nvidia_exit_module\n");
-    if (unregister_chrdev(nv_major, "nvidia") < 0)
-        nv_printf(NV_DBG_ERRORS, "NVRM: unregister nv chrdev failed\n");
+//		2.6.23 -- unregister_chrdev now returns void.  Don't ask why.
+    unregister_chrdev(nv_major, "nvidia") ;
     for (i = 0; i < num_nv_devices; i++)
         struct pci_dev *dev;

Now run nvidia-installer while booted into your lovely new 2.6.23 and all should be well.

Note: The patches to nv.c (the second block above) should compile just fine under any 2.6 kernel. The patch to nv-linux.h, though, changes the number of arguments to kmem_cache_create from 6 to 5, which will blow up with earlier kernels. There's a macro in the kernel source file slab.h to avoid exactly this problem, but for some reason Nvidia doesn't use it. (When I tried using it here I got an error blizzard and didn't want to take the time to find and fix the real problem.) So if you're going to be rebuilding older kernels, keep a copy of nv-linux.h with 6 arguments to the function in addition to the patched copy, and switch names or links appropriately. The new Nvidia driver (100.x) has a bit of cleverness in the make process that determines whether it needs 5 or 6 arguments, but I've just been manually switching a link when necessary.


14 November 2007

According to several correspondents (thanks!), the patch works for earlier versions of the Nvidia driver -- back into the 8xxxs -- if you tell patch to raise the fuzz factor a bit -- patch -F 3 < nvidia_whatever_patch

2.6.24-rcX -- unfortunately vesafb-tng will not compile into .24-series kernels (as of rc2, anyway); tng uses the zeromap_page_range function from memory.c (in /usr/src/linux/mm/). This function has disappeared in .24 and I am far too lazy cowardly to leap into the kernel's memory management functions to figure out what to use in tng instead. (I would love to post here patches for a version of vesafb-tng that will compile under .24; if any budding kernel guru wants to give it a try, I'll be happy to test it.) The good news, though, is that you can use the regular (old generation) vesa framebuffer in the kernel and the nvidia driver patched as above works fine. You have to put "vga=MagicModeNumber" into the kernel command line; appropriate magic numbers are listed in the kernel source Documentation/fb/vesafb.txt.