mmaping a kernel buffer to user space
in [Kernel]
|
From: Guillermo Marcus on 31 Oct 2006 04:20 Hi all, I recently run with the following situation while developing a PCI driver. The driver allocates memory for a PCI device using pci_alloc_consistent as this memory is going to be used to perform DMA transfers. To pass the data from/to the user application, I mmap the buffer into userspace. However, if I try to use remap_pfn_range (>=2.6.10) or the older remap_page_range(<=2.6.9) for mmaping, it ends up creating a new buffer, because they do not support RAM mapping, then pagefaulting to the VMA and by default allocating new pages. Therefore, I had to implement the nopage method and mmap one page at a time as they fault. However, to my point of view, this is unnecessary. The memory is already allocated, the memory is locked because it is consistent, and it may be a (very small) performance and stability issue to do them one-by-one. Why can't I simply mmap it all at once? am I missing some function? More important, why can't remap_{pfn/page}_range handle it? Best wishes, Guillermo Marcus Note: I am using kernel 2.6.9 for these tests, as it is required by my current setup. Maybe this issue has already been addressed in newer kernel. If that is the case, please let me know. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Jiri Slaby on 31 Oct 2006 11:10 Guillermo Marcus wrote: > Hi all, > > I recently run with the following situation while developing a PCI > driver. The driver allocates memory for a PCI device using > pci_alloc_consistent as this memory is going to be used to perform DMA > transfers. To pass the data from/to the user application, I mmap the > buffer into userspace. However, if I try to use remap_pfn_range > (>=2.6.10) or the older remap_page_range(<=2.6.9) for mmaping, it ends > up creating a new buffer, because they do not support RAM mapping, then > pagefaulting to the VMA and by default allocating new pages. Therefore, > I had to implement the nopage method and mmap one page at a time as they > fault. > > However, to my point of view, this is unnecessary. The memory is already > allocated, the memory is locked because it is consistent, and it may be > a (very small) performance and stability issue to do them one-by-one. > Why can't I simply mmap it all at once? am I missing some function? More > important, why can't remap_{pfn/page}_range handle it? Piece of code please. pci_alloc_consistent calls __get_free_pages, and there should be no problem with mmaping this area. regards, -- http://www.fi.muni.cz/~xslaby/ Jiri Slaby faculty of informatics, masaryk university, brno, cz e-mail: jirislaby gmail com, gpg pubkey fingerprint: B674 9967 0407 CE62 ACC8 22A0 32CC 55C3 39D4 7A7E - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Rolf Offermanns on 31 Oct 2006 11:20 Guillermo Marcus wrote: > I recently run with the following situation while developing a PCI > driver. The driver allocates memory for a PCI device using > pci_alloc_consistent as this memory is going to be used to perform DMA > transfers. To pass the data from/to the user application, I mmap the > buffer into userspace. However, if I try to use remap_pfn_range > (>=2.6.10) or the older remap_page_range(<=2.6.9) for mmaping, it ends > up creating a new buffer, because they do not support RAM mapping, then > pagefaulting to the VMA and by default allocating new pages. Therefore, > I had to implement the nopage method and mmap one page at a time as they > fault. > > However, to my point of view, this is unnecessary. The memory is already > allocated, the memory is locked because it is consistent, and it may be > a (very small) performance and stability issue to do them one-by-one. > Why can't I simply mmap it all at once? am I missing some function? More > important, why can't remap_{pfn/page}_range handle it? > Here is what I did some time ago: -> Reserve mem at boot time (mem=realmem-size_of_mem_you_need) / bigphysmem -> I used the highmem allocator from the LDD2/3 examples to get a pointer the this reserved memory at runtime. -> Use ioremap() to remap the memory to kernelspace -> do some magic (I don't remember the background, sorry) with the vma_flags in your mmap() function: vma->vm_flags |= VM_RESERVED; vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot); and then do a remap_pfn_range() as usualy. HTH, Rolf - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Guillermo Marcus on 31 Oct 2006 11:30 Hi Jiri, The fact that it does not works with RAM is well documented in LDD3, pages 430++. It says (and I tested) that remap_xxx_range does not work in this case. They suggest a method using nopage, similar to the one I implement. I do not see why remap_xxx_range has the limitation, but it is there. The question is then: can the limitation be removed, or can we implement a new function that maps RAM all at once without the need for a nopage implementation? In any case, here is the code. Best Wishes, Guillermo /*************************************************/ To allocate (inside an IOctl cmd): .... retptr = pci_alloc_consistent( privdata->pdev, kmem_handle->size, &(kmem_entry->dma_handle) ); if (retptr == NULL) goto kmem_alloc_mem_fail; kmem_entry->cpua = (unsigned long)retptr; kmem_entry->size = kmem_handle->size; kmem_entry->id = atomic_inc_return(&privdata->kmem_count) - 1; .... To mmap (inside mmap fops, this DOES NOT works): .... /* Map the Buffer to the VMA */ #if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,10) ret = remap_pfn_range( vma, vma->vm_start, __pa(kmem_entry->cpua) >> PAGE_SHIFT, kmem_entry->size, vma->vm_page_prot ); #else ret = remap_page_range( vma, vma->vm_start, __pa(kmem_entry->cpua), kmem_entry->size, vma->vm_page_prot ); #endif .... Forcing me to this: (in mmap): .... /* Set VM operations */ vma->vm_private_data = kmem_entry; vma->vm_ops = &pcidriver_vm_operations; pcidriver_vm_open(vma); .... (and the nopage vm_ops): .... /* Maps physical memory to user space */ struct page *pcidriver_vm_nopage(struct vm_area_struct *vma, unsigned long address, int * type) { pcidriver_kmem_entry_t *kmem_entry; struct page *page = NOPAGE_SIGBUS; unsigned long pfn_offset,pfn; /* Get private data for the page */ kmem_entry = vma->vm_private_data; /* All checks where done during the mmap_kmem call, so we can safely * just map the offset of the vm area to the offset of the region, * that is guaranteed to be contiguos */ pfn_offset = (address) - (vma->vm_start); pfn = (__pa(kmem_entry->cpua) + pfn_offset ) >> PAGE_SHIFT; if (!pfn_valid(pfn)) { mod_info("Invalid pfn in nopage() - 0x%lx \n", pfn); return NOPAGE_SIGBUS; } page = pfn_to_page( pfn ); get_page(page); if (type) *type = VM_FAULT_MINOR; return page; } .... /*************************************************/ Jiri Slaby wrote: > Guillermo Marcus wrote: >> Hi all, >> >> I recently run with the following situation while developing a PCI >> driver. The driver allocates memory for a PCI device using >> pci_alloc_consistent as this memory is going to be used to perform DMA >> transfers. To pass the data from/to the user application, I mmap the >> buffer into userspace. However, if I try to use remap_pfn_range >> (>=2.6.10) or the older remap_page_range(<=2.6.9) for mmaping, it ends >> up creating a new buffer, because they do not support RAM mapping, then >> pagefaulting to the VMA and by default allocating new pages. Therefore, >> I had to implement the nopage method and mmap one page at a time as they >> fault. >> >> However, to my point of view, this is unnecessary. The memory is already >> allocated, the memory is locked because it is consistent, and it may be >> a (very small) performance and stability issue to do them one-by-one. >> Why can't I simply mmap it all at once? am I missing some function? More >> important, why can't remap_{pfn/page}_range handle it? > > Piece of code please. pci_alloc_consistent calls __get_free_pages, and there > should be no problem with mmaping this area. > > regards, - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Guillermo Marcus on 31 Oct 2006 11:40 Hi Rolf, Thanks for your comments. Unfortunately, given some of the platforms we want to support, I cannot reserve memory at boot time, so I had to find some other way. Besides, it is anyway interesting to see how to present a buffer allocated using the DMA interfaces (pci_alloc or dma_alloc) to the user space. All the best, Guillermo Rolf Offermanns wrote: > Guillermo Marcus wrote: >> I recently run with the following situation while developing a PCI >> driver. The driver allocates memory for a PCI device using >> pci_alloc_consistent as this memory is going to be used to perform DMA >> transfers. To pass the data from/to the user application, I mmap the >> buffer into userspace. However, if I try to use remap_pfn_range >> (>=2.6.10) or the older remap_page_range(<=2.6.9) for mmaping, it ends >> up creating a new buffer, because they do not support RAM mapping, then >> pagefaulting to the VMA and by default allocating new pages. Therefore, >> I had to implement the nopage method and mmap one page at a time as they >> fault. >> >> However, to my point of view, this is unnecessary. The memory is already >> allocated, the memory is locked because it is consistent, and it may be >> a (very small) performance and stability issue to do them one-by-one. >> Why can't I simply mmap it all at once? am I missing some function? More >> important, why can't remap_{pfn/page}_range handle it? >> > Here is what I did some time ago: > > -> Reserve mem at boot time (mem=realmem-size_of_mem_you_need) / bigphysmem > -> I used the highmem allocator from the LDD2/3 examples to get a pointer > the this reserved memory at runtime. > -> Use ioremap() to remap the memory to kernelspace > -> do some magic (I don't remember the background, sorry) with the vma_flags > in your mmap() function: > > vma->vm_flags |= VM_RESERVED; > vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot); > > and then do a remap_pfn_range() as usualy. > > HTH, > Rolf > > > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo(a)vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
|
Next
|
Last
Pages: 1 2 3 4 Prev: usb device descriptor read/64, error -110 Next: Add IDE mode support for SB600 SATA |