Kernel stays alive by keeping a small reserve of memory (dozen or so pages). Most allocations are checked, and everything that can fail will fail before this reserve is touched. Point of the reserve is for kernel to survive until it's a deadlock anyway.
There's a service for userspace programs to map physical (unpaged) memory into their address spaces. This is available for boottime processes (drivers, realtime stuff). Boottime processes can give the capability to other processes as well, if they want. This part of the system is to be called "core processes" because it generally implements the base of the operating environment.
For the rest, there's the paged memory interface. Basic anything that implements two operations (fetch and store) can act as a backing store. Floppy driver qualifes, so does a suitable FTP program. The only limitation is that if you map memory from a certain backing store, you have to trust that store (up to a point, anyway). This "rest of the world" never gets any unpaged memory, but real memory bleeds into the system anyway, because core processes can also act as backing stores.
Long before kernel internal reserve is reached, another much larger threshold controls whether a page stealer runs. When the page stealer sees a page that hasn't been used recently, it unmaps it, and if the page isn't mapped in other processes, either frees it (clean) or sends to pager for storage (dirty). Pager will then hopefully free the page, but if it doesn't it will (eventually) be paged again, until it hits one of the core processes, and if that one fails to free anything, then the whole system can commit suicide and kernel panic, because nothing is going to work anyway.
