feeding a FIFO from PCI
in [Linux Hardware]
|
From: John Larkin on 12 Apr 2008 22:00 On Sat, 12 Apr 2008 11:59:13 -0700 (PDT), Didi <dp(a)tgi-sci.com> wrote: >John Larkin wrote: >> ... >>... Our box will output frequency sweeps, arbitrary >> waveforms, a couple of dozen voltages that can be changed/ramped per >> user desires, and some discrete logic levels and triggers. >> >> One architecture would pack an Intel-cpu SBC and a custom board in a >> 2U rack box. The SBC would talk gigabit ethernet to the customer's >> system and PCI to our board. >> >> ... >> ... >> >> 5. Other ideas? >> > >Since the echo you are getting so far indicates the latency using a >x86 >may well be too high under linux or whatever, I can suggest doing some >tiny >DPS thing for you - with PCI and Ethernet. The latency then is no >issue, >tcp/ip etc. comes with it, filesystem/disk etc. If you settle for 100 >MbpS Ethernet, >it is quite easy for me - I can reuse some of the MPC5200 designs I >have. >1 GbpS it will take some other part and more than 3-4 months, though. >I am not sure I can beat the cost&time of someone writing the thing >for you >under linux while I do the whole thing, but I am willing to try hard >to do so, >the time has come when I want to make all that stuff I have more >popular than it is now. > We could put a powerQuicc or a Blackfin on the board. But then we'd need dram for the sequence storage, or we'd have to interface the cpu's ram to the fpga some fast way, and we'd have to do the gigabit ethernet and the tcp/ip stack and all that. We can get that stuff, already done, with a 2 GHz dual-core CPU, for under $400. The sbc has a lot of stuff already done. It will run Linux the day we open the box. John
From: rickman on 12 Apr 2008 22:00 On Apr 12, 8:46 pm, hal-use...(a)ip-64-139-1-69.sjc.megapath.net (Hal Murray) wrote: > >I would again suggest that you can *simplify* the project by putting > >RAM on your board instead of a *real time* PCI interface. > > But he doesn't need any real time software. All he needs > is a big buffer in memory. A DMA engine on the card will > grab data from memory whenever the FIFO has room. > > >You can add a GB of DRAM to your board that with just a very few > >chips, I haven't looked at the available RAM chips lately but I > >believe they are beyond the Gbit level. Or you can use a module and > >plug in what ever size you need. You can lose the PCI interface by > >using something serial which can be done with a single MCU chip. > >Check out the Luminary Micro parts with Ethernet including the PHY. > >PCI is not really all that fast and getting any real speed out of it > >will take a fair amount of programming effort. > > Yes, a PC is overkill for just grabbing the data and buffering it > for the FPGA. There may well be better overall designs that don't > use a PC or PCI. > > On the other hand, the PCI part of the board design is only ~40 or > ~70 wires. I think the PCI logic is roughly as complicated as the > DRAM interface logic. (handwave) For me the issue is not the complexity of the hardware because I think that is in the noise for this project. The issue is the complexity of interfacing an FPGA to a bank of memory and to an MCU which has either USB or Ethernet connectivity compared to the complexity of interfacing an FPGA to a PCI bus and developing the software to support whatever transactions will be happening over the PCI bus. I guess if you have designed PCI bus DMA hardware and software before, then this is not a real issue. The experience I had was that the hardware for the FPGA and DRAM was done and working 100% on schedule. The software had significant complications and was the limiting factor in the project schedule. When you say that you don't need "real time software", I am missing something. Once started, does the DMA run to completion by itself? Maybe I am not up to speed with current software techniques on the PC, but I thought even DMA required real time response to keep it queued up and running. As far as allocating a block of memory to buffer the data, I have no understanding of what it takes to allocate a buffer of half a GB or more of contiguous memory. But like I said, I am not so familiar with this approach. I am, however, familiar with memory interfaces. They are well specified in maybe a dozen pages vs. the hundreds of pages for the PCI bus and the virtually unlimited amount of documentation (or lack thereof) for the operating system and writing drivers for DMA. To me the issue is that even if the DRAM hardware is about the same complexity as the PCI bus hardware, it just seems like everything else is a lot less complex by offloading the memory buffer onto the board. The hard parts of this project are the real time issues. I just seems so much simpler to keep all of the real time aspects on the board *in 100% controllable hardware* and AWAY from the Intel CPU, the shared PCI bus, DMA controllers and some rather arcane software.
From: Hal Murray on 12 Apr 2008 23:24 >When you say that you don't need "real time software", I am missing >something. Once started, does the DMA run to completion by itself? >Maybe I am not up to speed with current software techniques on the PC, >but I thought even DMA required real time response to keep it queued >up and running. As far as allocating a block of memory to buffer the >data, I have no understanding of what it takes to allocate a buffer of >half a GB or more of contiguous memory. But like I said, I am not so >familiar with this approach. The basic idea is that you give the FPGA a pointer and length. It reads memory a cache block at a time as it needs it. When it's done, it sets a status bit and maybe generates an interrupt. The only thing that's different with this design and a typical disk or network transfer is that this one will be much larger. You might have to give it a clump of pointer/length pairs, either stored in memory or on chip. You could give it each piece of the clump one at a time, but that gets you into the time constraints. I haven't actually written the code (driver or FPGA) to do this. I've worked on projects that did similar things. It's possible I'm overlooking something critical. Maybe allocating huge (as compared to big) chunks of memory is hard. I'm sure a good kernel wizard can do it one way or the other. If nothing else, you hack the very early part of the kernel to put some memory in it's back pocket until you ask for it. Ugly, but effective. -- These are my opinions, not necessarily my employer's. I hate spam.
From: Didi on 12 Apr 2008 23:36 John Larkin wrote: > .... > We could put a powerQuicc or a Blackfin on the board. But then we'd > need dram for the sequence storage, or we'd have to interface the > cpu's ram to the fpga some fast way, and we'd have to do the gigabit > ethernet and the tcp/ip stack and all that. We can get that stuff, > already done, with a 2 GHz dual-core CPU, for under $400. > > The sbc has a lot of stuff already done. It will run Linux the day we > open the box. > I know PCs are cheap. But you are after something more - which will take some programming and latency spec meeting. I have no idea how viable the thing is and how much time & cash it will cost you. If I were to do it on a 5200 or other similar part I would likely do the actual Internet --> FIFO thing within a few days; having DPS run on the particular platform is hard to be predicted, if a 5200 is used a week to a month, I would say. Dimiter ------------------------------------------------------ Dimiter Popoff Transgalactic Instruments http://www.tgi-sci.com ------------------------------------------------------ http://www.flickr.com/photos/didi_tgi/sets/72157600228621276/ Original message: http://groups.google.com/group/comp.arch.embedded/msg/9a636587b48980f3?dmode=source
From: Jure Newsgroups on 13 Apr 2008 15:48
"Hal Murray" <hal-usenet(a)ip-64-139-1-69.sjc.megapath.net> wrote in message news:KNOdnTzMponprpzVnZ2dnUVZ_v3inZ2d(a)megapath.net... > >>One architecture would pack an Intel-cpu SBC and a custom board in a >>2U rack box. The SBC would talk gigabit ethernet to the customer's >>system and PCI to our board. > > 2U is ugly in that you can't get full height PCI cards without > using a riser kludge to turn the card sideways. > > I think PCI cards fit in 3U. There is a short size that fits in 2U. > (or you can cross your fingers on the riser stuff.) > > >>Our board would have a PCI interface driving a biggish FIFO, say 8k >>deep by 48 bits wide, inside an FPGA. A simple state machine/latch/mux >>thing repacks the 32-bit pci transfers into the input of the 48-bit >>wide fifo. The output side of the FIFO would be driving a fairly >>simple state machine; each fifo word has an opcode field and a data >>field, with different opcodes feeding various devices connected to the >>physics... dds synthesizers, ttl outputs, whatever. The state machine >>that unloads the fifo would run at 128 MHz, but one opcode is WAIT, so >>we can slow down operations to match the realtime needs of the >>experiment and reduce the average fifo feed rate. > >>OK, we finally get to a question: If we run some flavor of Linux on >>the SBC, what's a good strategy for keeping the fifo loaded? Assuming >>that we have the recipe for an entire experimental shot in program >>ram, some tens of megabytes maybe, we could... > > >>3. Best, if possible: set up a single DMA transfer to do the entire >>shot. That involves a dma controller that understands that the target >>is sometimes busy, and retries after getting bounced. I know the pci >>bus has hooks for split transfers, but I don't know if standard >>Intel-type dma controllers can work in this mode. > > I think that's what you want to do. It comes for free. I think > it will all make sense if you read the PCI specs. Or maybe > just the specs for the PCI interface block you are going to use. > > Ignoring pipeline problems, the host side of a DMA read request > doesn't know how how much data the device wants. It just gets > an op-code that says read or read-cache-line. Once data > starts flowing, either side can say I'm-done-now. If the device > (still) wants more data, it starts over with bus arbitration. > The host may say "done" to let another device have a turn > or to cross a page boundary or ... > > > The DMA section of the FPGA will run in chatter mode. When > there is room in the FIFO for another cache block, it will > ask for more data. When the FIFO is near-full, it will stop > asking. You have to leave enough room in the FIFO to hold all > the data in the pipeline. > > > One quirk. The driver has to allocate a chunk of physically > contigious memory. That probably has to happen early in the > boot-up time so you still have a chunk of contigious memory > to grab. > > -- > These are my opinions, not necessarily my employer's. I hate spam. > would this product work ? http://www.strategic-test.com/ultrafast2_pci-x_cards/uf2-7000.htm I'll check in my files at work for other possible product I have seen before. Thanks, Jure Z. |