Prev: Modify UCF file generated with MIG
Next: how to store data in i2c slave
From: Frank van Eijkelenburg on 2 Aug 2010 11:10
Hi,

I have a custom made PCIe board with a Virtex 5 FPGA on which I
implemented a DMA unit which uses the PCIe endpoint block plus v1.14.
I also implemented simple read/write operations from the PC to the
board (the board responds with completion TLPs). The read/write
operations are working, DMA is not working (transferring data from
FPGA to PC).

The board is inserted in a pc with Windows 7 64 bits platform. An
application allocates virtual memory and passes a pointer to the
memory block to the driver. The driver locks the memory pages and
creates a scatter-gather list with physical addresses by using the DMA
adapter structure. The physical addresses are written to the FPGA.

When I start a DMA operation by writing a register in the FPGA, I can
see in chipscope the correct physical addresses in the TLP header (of
the memory write requests). However, I do not see the correct values
in the allocated memory at the PC. What can I do to check where it is
going wrong?

In my opinion there are two possibilities: either the TLP is blocked
by a PCIe switch at the main board or the data is available at another
memory location.

I also tried the other direction (send memory read requests TLPs from
the FPGA to the PC and receive completion TLPs as answer at the memory
read request). In the completion TLPs I see the correct data (data I
wrote into the PC memory before starting the DMA operation).

Any suggestions/ideas are welcome.

Thanks in advance,

Frank
From: glen herrmannsfeldt on 2 Aug 2010 11:28
Frank van Eijkelenburg <fei.technolution(a)gmail.com> wrote:

> I have a custom made PCIe board with a Virtex 5 FPGA on which I
> implemented a DMA unit which uses the PCIe endpoint block plus v1.14.
> I also implemented simple read/write operations from the PC to the
> board (the board responds with completion TLPs). The read/write
> operations are working, DMA is not working (transferring data from
> FPGA to PC).
(snip)

> When I start a DMA operation by writing a register in the FPGA, I can
> see in chipscope the correct physical addresses in the TLP header (of
> the memory write requests). However, I do not see the correct values
> in the allocated memory at the PC. What can I do to check where it is
> going wrong?

Not having tried to do DMA through PCI before, is data being
written, but the wrong data?

I would try writing all zeros or all ones and see if those come
through fine. It could be timing between the FPGA and PCI such
that the wrong data is being latched.

Then try slightly less predictable data and see what gets through.

-- glen
From: Frank van Eijkelenburg on 3 Aug 2010 03:26
On Aug 2, 5:28 pm, glen herrmannsfeldt <g...(a)ugcs.caltech.edu> wrote:
> Frank van Eijkelenburg <fei.technolut...(a)gmail.com> wrote:
>
> > I have a custom made PCIe board with a Virtex 5 FPGA on which I
> > implemented a DMA unit which uses the PCIe endpoint block plus v1.14.
> > I also implemented simple read/write operations from the PC to the
> > board (the board responds with completion TLPs). The read/write
> > operations are working, DMA is not working (transferring data from
> > FPGA to PC).
>
> (snip)
>
> > When I start a DMA operation by writing a register in the FPGA, I can
> > see in chipscope the correct physical addresses in the TLP header (of
> > the memory write requests). However, I do not see the correct values
> > in the allocated memory at the PC. What can I do to check where it is
> > going wrong?
>
> Not having tried to do DMA through PCI before, is data being
> written, but the wrong data?

That is what I do not know. Yes the correct data is send to the PC,
but if I readout the memory the values are unchanged.

>
> I would try writing all zeros or all ones and see if those come
> through fine.  It could be timing between the FPGA and PCI such
> that the wrong data is being latched.
>
> Then try slightly less predictable data and see what gets through.
>
> -- glen


If it was timing, I expect the other way around also problems (which I
don't have). Also single memory read/write requests send from the PC
are working correctly.

From: alpha on 3 Aug 2010 12:03
On Aug 3, 3:26 am, Frank van Eijkelenburg <fei.technolut...(a)gmail.com>
wrote:
> On Aug 2, 5:28 pm, glen herrmannsfeldt <g...(a)ugcs.caltech.edu> wrote:
>
>
>
> > Frank van Eijkelenburg <fei.technolut...(a)gmail.com> wrote:
>
> > > I have a custom made PCIe board with a Virtex 5 FPGA on which I
> > > implemented a DMA unit which uses the PCIe endpoint block plus v1.14.
> > > I also implemented simple read/write operations from the PC to the
> > > board (the board responds with completion TLPs). The read/write
> > > operations are working, DMA is not working (transferring data from
> > > FPGA to PC).
>
> > (snip)
>
> > > When I start a DMA operation by writing a register in the FPGA, I can
> > > see in chipscope the correct physical addresses in the TLP header (of
> > > the memory write requests). However, I do not see the correct values
> > > in the allocated memory at the PC. What can I do to check where it is
> > > going wrong?
>
> > Not having tried to do DMA through PCI before, is data being
> > written, but the wrong data?
>
> That is what I do not know. Yes the correct data is send to the PC,
> but if I readout the memory the values are unchanged.
>
>
>
> > I would try writing all zeros or all ones and see if those come
> > through fine. It could be timing between the FPGA and PCI such
> > that the wrong data is being latched.
>
> > Then try slightly less predictable data and see what gets through.
>
> > -- glen
>
> If it was timing, I expect the other way around also problems (which I
> don't have). Also single memory read/write requests send from the PC
> are working correctly.

Does CPU caches get snooped when DMA wrote into host memory? If not,
then you may not see right data by normal read.

From: sreekanth on 5 Aug 2010 11:44
AFAIK, in x86 architectures, io writes will be cache coherent, unless you're setting SNR (no snoop) in the write header by mistake

But to me this sounds like a Windows problem. Typically garden variety PC's don't block PCIE accesses, all of the address space is open to a peripheral. Acid test : if you can write to a given physical address from your FPGA, and read the value, you're golden in the physical address domain. So maybe what you think you're reading in your code is not actually getting to the correct physical address.

try :

write from FPGA card to a physical address
read back using FPGA card, see if good.
write some known data from windows software
read back using FPGA card.


 |  Next  |  Last
Pages: 1 2
Prev: Modify UCF file generated with MIG
Next: how to store data in i2c slave