mmap on MacOSX

Programming, for all ages and all languages.
Post Reply
User avatar
blueshogun96
Posts: 2
Joined: Wed Sep 04, 2013 3:32 pm
Location: Seattle

mmap on MacOSX

Post by blueshogun96 »

Hello OSDev.org! I've been searching and asking all over the internet for a solution to this problem and I haven't found one yet.

I am trying to get access to a *very* specific memory address within the first 4GB of address space using a 64-bit application in MacOSX. Ideally, the first 64mb at least. On Windows and Linux, I've been able to accomplish this (Linux was easy, of course, but Windows took a bit more work), but absolutely cannot get this to work on MacOSX! I assumed that since MacOSX is a UNIX based OS, it would be as simple as this (which works fine on Linux btw):

Code: Select all

#include <sys/mman.h>
#include <iostream>

namespace {
    void * const MEMORY_ADDRESS = reinterpret_cast<void *>(0x10000);
    size_t const MEMORY_SIZE = 1 << 20;
}

int main() {
    void * p = mmap(MEMORY_ADDRESS, MEMORY_SIZE, PROT_EXEC | PROT_READ | PROT_WRITE, MAP_FIXED | MAP_PRIVATE | MAP_ANONYMOUS, 0, 0);
    if (p != reinterpret_cast<void *>(-1)) {
        std::cout << "Memory allocated" << std::endl;
        munmap(p, MEMORY_SIZE);
    } else {
        std::cout << "Memory could not be allocated" << std::endl;
    }
}
Doesn't work on OSX. I heard that the first 4GB of address space was unused in 64-bit OSX. Is that wrong? My knowledge of MacOSX's memory map layout is minimal, and there isn't much documentation on how this OS works.

In order for this to work on Windows, I had to *trick* Windows into letting me have it by setting the base address of the .exe, declare a fixed array of bytes the length of the memory range I want to reserve, put all of my Win32 code into a .dll and *only* call the function from the .exe, set the permissions using VirtualProtect, back up the every byte that the .exe has taken up in that memory address range, and then do my dirty work until it's time to exit, in which I restore the .exe contents, and return from the .dll function.

You might be thinking "why on earth do you need to do this??". It's for a VM I'm writing, without the need for VMX (because VMX seems like a pain to use). If you need a more detailed description of what I'm trying to do, feel free to ask anything specific. Thanks.

- Shogun.
User avatar
sortie
Member
Member
Posts: 931
Joined: Wed Mar 21, 2012 3:01 pm
Libera.chat IRC: sortie

Re: mmap on MacOSX

Post by sortie »

Hmm, I don't have a solution, but you may wish to compare the mmap return value against MAP_FAILED (which is -1, but that's not standardized last I checked!). Additionally, the fd argument to mmap should be -1 (or something invalid) when MAP_ANONYMOUS is given.
h0bby1
Member
Member
Posts: 240
Joined: Wed Aug 21, 2013 7:08 am

Re: mmap on MacOSX

Post by h0bby1 »

i'm not very familliar with macos, and there is very little documentation avaible, but the macos kernel (darwin ?) is supposed to be open source no ? maybe you can check how the mmap function work, or you can try to step into the kernel code with a debugger to see where it fail and why
User avatar
blueshogun96
Posts: 2
Joined: Wed Sep 04, 2013 3:32 pm
Location: Seattle

Re: mmap on MacOSX

Post by blueshogun96 »

sortie wrote:Hmm, I don't have a solution, but you may wish to compare the mmap return value against MAP_FAILED (which is -1, but that's not standardized last I checked!). Additionally, the fd argument to mmap should be -1 (or something invalid) when MAP_ANONYMOUS is given.
Point taken, but that also reminds me... I forgot to mention the error code returned from errno (aka the __error() function); ENOMEM (12). I don't know if the memory range is already in use, or if the kernel just denies access to it for security reasons.

I also changed the fd argument to -1, but unfortunately that didn't work either.
h0bby1 wrote:i'm not very familliar with macos, and there is very little documentation avaible, but the macos kernel (darwin ?) is supposed to be open source no ? maybe you can check how the mmap function work, or you can try to step into the kernel code with a debugger to see where it fail and why
Last I checked (since 10.5) it is. If Darwin is open source (the OSX version, I assume), then I'd assume that Apple has done various modifications to it for their own security reasons. Checking the source code wouldn't be a bad idea, and stepping into the kernel code would be even better, however, I never thought that I could debug kernel level code using XCode. If that's really possible, then I will gladly dig a bit deeper.

Another thing I should have mentioned before (sorry, I forgot) is that "/dev/mem" access was fairly recently disabled in MacOSX for security reasons IIRC. It's been a while since I looked into this issue, and I heard it's possible to re-enable it somehow. The reason why it fails might be linked to this little exclusion by Apple. I think I might have asked about this on a Mac oriented forum, but I don't recall getting and responses though.

Thanks for the responses. Any more ideas?

- Shogun.
h0bby1
Member
Member
Posts: 240
Joined: Wed Aug 21, 2013 7:08 am

Re: mmap on MacOSX

Post by h0bby1 »

i don't know if you can step into the code , it depend where the interesting part of the code reside, and how the call is made, if it use an interupt or task switch to make the call, it might be difficult to get to the actual code, but it can worth a try, i already stepped in many system libraries under windows, but i think there is a point where the debugger say you can't go further, but maybe it's specific to visual studio and windows

as there is little documentation for mac os , either you can try to find a macos guru who have figured this out, or trying to ask directly to apple in a way or another, they have a developper center and maybe you can grab some mail or contact information to ask specific questions to macos developpers, sometime they might answer

otherwise you can only try to look in kernel source it might give some hints on how the mmap is supposed to work, even if the kernel used is not exactly compiled from them, it can give you an idea of things you might be doing wrong, or even possibly to find other way to access the memory with other functions
User avatar
Brynet-Inc
Member
Member
Posts: 2426
Joined: Tue Oct 17, 2006 9:29 pm
Libera.chat IRC: brynet
Location: Canada
Contact:

Re: mmap on MacOSX

Post by Brynet-Inc »

h0bby wrote:as there is little documentation for mac os , ..
While I'm certainly not vouching for the quality or accuracy of Apple's documentation, that isn't exactly truthful. They've written man pages for original utilities and do document at least some changes made to stuff imported from BSD and GNU. They also maintain their own documentation on their Developer Library site.

https://developer.apple.com/library/mac ... map.2.html

Perhaps the mmap() problem is related to their recent introduction of ASLR (Address Space Layout Randomization); the source code is likely a better reference IMHO.

There is also no evidence to suggest the source code is significantly different from the binaries they release, it's certainly possible that they omit some proprietary components.. but the "Hackintosh" community uses it as a basis for running on non-Apple branded systems, from only gleaning info it seems their primary issue is hardware support.

http://www.opensource.apple.com/source/ ... ern_mman.c
http://www.opensource.apple.com/tarball ... .15.tar.gz

Just a heads up Shogun, most members here are discussing hobby OS development.. questions about mainstream operating systems like Linux or Mac OS X are strongly discouraged here (..there are other communities).
Image
Twitter: @canadianbryan. Award by smcerm, I stole it. Original was larger.
User avatar
dozniak
Member
Member
Posts: 723
Joined: Thu Jul 12, 2012 7:29 am
Location: Tallinn, Estonia

Re: mmap on MacOSX

Post by dozniak »

You can build your own Darwin and replace OSX kernel with it and it should work.

You can build it with debug info and whatnot and debug in Xcode or plain lldb. The kernel is pretty small, and most functionality is contained in proprietary modules anyway.
Learn to read.
User avatar
Owen
Member
Member
Posts: 1700
Joined: Fri Jun 13, 2008 3:21 pm
Location: Cambridge, United Kingdom
Contact:

Re: mmap on MacOSX

Post by Owen »

$ man 1 ld

Code: Select all

     -pagezero_size size
                 By default the linker creates an unreadable segment starting
                 at address zero named __PAGEZERO.  Its existence will cause a
                 bus error if a NULL pointer is dereferenced.  The argument
                 size is a hexadecimal number with an optional leading 0x.  If
                 size is zero, the linker will not generate a page zero seg-
                 ment.  By default on 32-bit architectures the page zero size
                 is 4KB.  On 64-bit architectures, the default size is 4GB.
                 The ppc64 architecture has some special cases. Since Mac OS X
                 10.4 did not support 4GB page zero programs, the default page
                 zero size for ppc64 will be 4KB unless -macosx_version_min is
                 10.5 or later.  Also, the -mdynamic-no-pic codegen model for
                 ppc64 will only work if the code is placed in the lower 2GB
                 of the address space, so the if the linker detects any such
                 code, the page zero size is set to 4KB and then a new unread-
                 able trailing segmen
Also, your windows approach will explode the minute something raises or catches a Win32 SEH exception. Which it will. If not on your machine, on somebody else' machine. Don't do it. Just pass the address you need to VirtualAlloc instead.

Though you have yet to give a good reason why you need fixed virtual addresses. Whatever you're planning, either you're misunderstanding VMX and SVM, or you're doing things the hard way. Compared to the other methods, VMX and SVM are trivial.
Gigasoft
Member
Member
Posts: 855
Joined: Sat Nov 21, 2009 5:11 pm

Re: mmap on MacOSX

Post by Gigasoft »

blueshogun96 wrote:In order for this to work on Windows, I had to *trick* Windows into letting me have it by setting the base address of the .exe, declare a fixed array of bytes the length of the memory range I want to reserve, put all of my Win32 code into a .dll and *only* call the function from the .exe, set the permissions using VirtualProtect, back up the every byte that the .exe has taken up in that memory address range, and then do my dirty work until it's time to exit, in which I restore the .exe contents, and return from the .dll function.
There's another, perhaps better way to do this, but you need to create a second process using ZwCreateUserProcess with a special parameter. This is the prototype for ZwCreateUserProcess:

Code: Select all

NTSTATUS
  ZwCreateUserProcess (
    __out PHANDLE  ProcessHandle,
    __out PHANDLE  ThreadHandle,
    __in DWORD  ProcessDesiredAccess,
    __in DWORD  ThreadDesiredAccess,
    __in POBJECT_ATTRIBUTES ProcessObjectAttributes,
    __in POBJECT_ATTRIBUTES ThreadObjectAttributes,
    __in DWORD  ProcessFlags,
    __in DWORD  ThreadFlags
    __in RTL_USER_PROCESS_PARAMETERS  UserProcessParameters,
    __in CREATEPROCESS_PARAMS*  Parameters,
    __inout EXTENDED_PARAMETERS*  ExtendedParameters
  );
UserProcessParameters is laid out like this:

00000000 MaximumLength dd ?
00000004 Length dd ?
00000008 Flags dd ?
0000000C DebugFlags dd ?
00000010 ConsoleHandle dd ?
00000014 ConsoleFlags dd ?
00000018 InputHandle dd ?
0000001C OutputHandle dd ?
00000020 ErrorHandle dd ?
00000024 CurrentDirectory CURDIR ? ; path (UNICODE_STRING), handle
00000030 DllPath UNICODE_STRING ?
00000038 ImagePathName UNICODE_STRING ?
00000040 CommandLine UNICODE_STRING ?
00000048 Environment dd ?
0000004C StartingX dd ?
00000050 StartingY dd ?
00000054 CountX dd ?
00000058 CountY dd ?
0000005C CountCharsX dd ?
00000060 CountCharsY dd ?
00000064 FillAttribute dd ?
00000068 WindowFlags dd ?
0000006C ShowWindowFlags dd ?
00000070 WindowTitle UNICODE_STRING ?
00000078 DesktopInfo UNICODE_STRING ?
00000080 ShellInfo UNICODE_STRING ?
00000088 RuntimeData UNICODE_STRING ?
00000090 DLCurrentDirectory RTL_DRIVE_LETTER_CURDIR 32 dup(?) ; flags (word), length (word), timestamp (dword), path (UNICODE_STRING)
00000290 EnvironmentLength dd ?
00000294 field_294 dd ?
[/code]

Parameters is a 72 byte input/output buffer. You fill it with: size (dword: 72), information class (dword: 0), flags1 (byte), flags2 (byte), incompatible characteristics (word), executable access mask (dword). On return, it contains various other information.

ExtendedParameters consists of a size field followed by one or more parameters consisting of an ID, the buffer size, a pointer to the parameter buffer and a pointer to a variable which receives the returned length (must be 0 for input parameters). You need the following parameters:
0x20005 - executable filename (unicode string)
0x20007 - reserved virtual address ranges: base address (dword), size (dword), can hold multiple entries

Other possible parameters are:
0x60000 - specifies parent process
0x60001 - specifies debug object
0x60002 - specifies primary token
0x10003 - returns CLIENT_ID
0x10004 - returns TEB address
0x00006 - returns SECTION_IMAGE_INFORMATION for the executable
0x20008 - specifies base priority
0x20009 - specifies hard error mode
0x2000a - specifies flags (bits 0-1: 0=don't use console handles, 1=inherit console handles if same subsystem, 2=use console handles from UserProcessParameters, bit 2: inherit stdin, bit 3: inherit stdout, bit 4: inherit stderr)
0x2000b - specifies an array of handles to inherit
Post Reply