ruby bounties--list of bounties
in [Ruby]
|
From: Mike Dalessio on 25 Jan 2010 14:34 [Note: parts of this message were removed to make it a legal post.] On Mon, Jan 25, 2010 at 2:17 PM, Chuck Remes <cremes.devlist(a)mac.com> wrote: > > On Jan 25, 2010, at 1:12 PM, Mike Dalessio wrote: > > > On Mon, Jan 25, 2010 at 1:12 PM, Charles Oliver Nutter > > <headius(a)headius.com>wrote: > > > >> On Mon, Jan 25, 2010 at 6:25 PM, Mike Dalessio <mike.dalessio(a)gmail.com > > > >> wrote: > >>> Charlie, you're making a great case against using FFI. > >> > >> FFI is much better than writing any C code at all, due to the > >> security, stability, and portability problems of writing your own C > >> bindings. If you are permitted to load a given library and that > >> library is available and you *must* use that library, FFI is the only > >> logical choice. But it doesn't get around the fact that you need the > >> library you're binding to be available and loadable on your target > >> platform. FFI > C bindings, but [platform-independent binary] > FFI. > >> And that usually means Java-based. > >> > >> I should also point out that you don't necessarily have to write JVM > >> libraries in Java; you could also use Scala or Fan or similar > >> languages, and it would be just as portable (albeit a bit larger due > >> to the runtime dependency on those languages' runtime libraries). > >> > >> But yes, at the end of the day, I believe writing stuff in a portable > >> binary format like JVM bytecode (or CLR bytecode) is a better choice > >> than writing in a language that has to be recompiled for every target > >> system. You ought to know that already...would I be working on JRuby > >> if I believed any differently? :) > >> > > > > I agree with everything you're saying, more or less. > > > > However, none of that relates at all to what I think is the crux of the > > issue, which is that everyone writing a non-pure-Ruby gem today is forced > to > > choose one of these options: > > > > 1) Support nearly everyone by maintaining two ports of your code: FFI for > > JRuby; C for MRI, Rubinius and MacRuby. Don't support GAE. > > 2) Support everyone by maintaining two ports of your code: JVM for JRuby > and > > GAE; C for MRI, Rubinius and MacRuby. > > 3) Maintain only a single port, FFI, and force everyone on MRI to take a > > performance hit of some kind. Oh, and don't support Rubinius, MacRuby or > > GAE. > > 4) Don't support JRuby or GAE. Just write it in C. > > 5) Don't support MRI, Rubinius, or MacRuby. Just write it for the JVM. > > FFI originated with rubinius, so I would wager that it will work once the > FFI APIs get synched up again. Also, MacRuby has FFI support on its roadmap. > That changes your picture a bit. > If you're interested in helping out in standardizing the FFI specs, please subscribe to the ruby-ffi list and offer to help out! We're always looking for extra hands, because the specs are not in good shape right now. So I'm likely to take your wager. ;) I stand by the chart as an accurate reflection of the options that developers are forced to choose from today and for the likely near future. > > cr > > > -- mike dalessio mike(a)csa.net
From: Chuck Remes on 25 Jan 2010 14:52 [Note: parts of this message were removed to make it a legal post.] On Jan 25, 2010, at 1:34 PM, Mike Dalessio wrote: > On Mon, Jan 25, 2010 at 2:17 PM, Chuck Remes <cremes.devlist(a)mac.com> wrote: >> >> FFI originated with rubinius, so I would wager that it will work once the >> FFI APIs get synched up again. Also, MacRuby has FFI support on its roadmap. >> That changes your picture a bit. >> > > If you're interested in helping out in standardizing the FFI specs, please > subscribe to the ruby-ffi list and offer to help out! We're always looking > for extra hands, because the specs are not in good shape right now. So I'm > likely to take your wager. ;) > > I stand by the chart as an accurate reflection of the options that > developers are forced to choose from today and for the likely near future. While it may be true that *some* C extensions work with rubinius and MacRuby today, I'd say it doesn't matter much in the long term. For one, Rubinius does not support the entire MRI C API nor will it ever. Extensions that directly access memory structures are not supported. FFI is a better long-term choice for Rubinius. MacRuby is months away from catching up to Rubinius, JRuby or IronRuby for handling straight ruby code. I don't mean to disparage MacRuby (it will likely be my go-to-guy for future Cocoa apps) but it ain't ready for prime time for *ruby* code let alone hooking in C extensions. And like Rubinius, it won't support all of the MRI C API. IronRuby does not support any C extensions though it's on the roadmap. I don't know for certain how extensive their support will be, but I will *wager* they'll avoid supporting the same elements that Rubinius and MacRuby are avoiding. :) So for the likely near future (next 6 months), Rubinius is the only one that might be able to run a random C extension (as long as it doesn't use unsafe direct access to memory structures). I understand what you are saying, truly I do. But I disagree that it is important to continue building extensions using the C API for the *long* term. The best way to get FFI firmed up and ready for prime-time is to port existing extensions to it. cr
From: Aaron Patterson on 25 Jan 2010 15:56 On Tue, Jan 26, 2010 at 04:52:13AM +0900, Chuck Remes wrote: > > On Jan 25, 2010, at 1:34 PM, Mike Dalessio wrote: > > > On Mon, Jan 25, 2010 at 2:17 PM, Chuck Remes <cremes.devlist(a)mac.com> wrote: > >> > >> FFI originated with rubinius, so I would wager that it will work once the > >> FFI APIs get synched up again. Also, MacRuby has FFI support on its roadmap. > >> That changes your picture a bit. > >> > > > > If you're interested in helping out in standardizing the FFI specs, please > > subscribe to the ruby-ffi list and offer to help out! We're always looking > > for extra hands, because the specs are not in good shape right now. So I'm > > likely to take your wager. ;) > > > > I stand by the chart as an accurate reflection of the options that > > developers are forced to choose from today and for the likely near future. > > While it may be true that *some* C extensions work with rubinius and MacRuby today, I'd say it doesn't matter much in the long term. > > For one, Rubinius does not support the entire MRI C API nor will it ever. Extensions that directly access memory structures are not supported. FFI is a better long-term choice for Rubinius. It doesn't need to support the entire API. It supports enough of the C API to get nokogiri running, and believe me, we use a *lot* of the C API. Why pay the FFI speed penalty when you can write C code that works cross implementation? > MacRuby is months away from catching up to Rubinius, JRuby or IronRuby for handling straight ruby code. I don't mean to disparage MacRuby (it will likely be my go-to-guy for future Cocoa apps) but it ain't ready for prime time for *ruby* code let alone hooking in C extensions. And like Rubinius, it won't support all of the MRI C API. Again, it doesn't need to support the entire C api. > IronRuby does not support any C extensions though it's on the roadmap. I don't know for certain how extensive their support will be, but I will *wager* they'll avoid supporting the same elements that Rubinius and MacRuby are avoiding. :) > > So for the likely near future (next 6 months), Rubinius is the only one that might be able to run a random C extension (as long as it doesn't use unsafe direct access to memory structures). > > I understand what you are saying, truly I do. But I disagree that it is important to continue building extensions using the C API for the *long* term. The best way to get FFI firmed up and ready for prime-time is to port existing extensions to it. As I pointed out in an earlier email, dealing with FFI wrapped libraries is error prone, difficult to debug (not just during development, but also when helping people get things installed), doesn't work cross implementation, requires id2ref (the bane of Charlie's existence. I'm sorry. :-( ), etc. I even have real world examples of *all* of the issues I pointed out. Even if FFI were the cross implementation messiah it's supposed to be, our FFI applications will *still* not work on GAE or Android. Rubinius has already proved that you can implement a *subset* of the C API and get complex extensions to work. Why can't we run with that? I think it would be a better long term solution. We would get the same "cross implementation" behavior as FFI, but not have to pay FFI's runtime conversion penalties. We also get the ability to do compile time checks of C library functionality (i.e. check for #defines, function existence, etc). People keep saying that FFI is the better way to go, but as someone who has to support both an FFI version and a C version, I can tell you the support / development problems with FFI are much more difficult. -- Aaron Patterson http://tenderlovemaking.com/
From: Chuck Remes on 25 Jan 2010 16:20 On Jan 25, 2010, at 2:56 PM, Aaron Patterson wrote: > People keep saying that FFI is the better way to go, but as someone who > has to support both an FFI version and a C version, I can tell you the > support / development problems with FFI are much more difficult. 1. I have no direct experience using FFI, so my opinions should carry the appropriate weight. I defer to your real-world experience. 2. I'm not much of a bikeshedder. I agree with Eleanor. Let's fix the performance deficiencies in the runtimes and write more code in ruby. cr
From: Charles Oliver Nutter on 26 Jan 2010 02:58
On Mon, Jan 25, 2010 at 7:53 PM, Aaron Patterson <aaron(a)tenderlovemaking.com> wrote: > References please. > > Last I checked, it was just as easy to segv from an FFI library as a C > library. Â Plus with FFI you don't get any benefits of compile time > checks. Â You can't, for example, check for #define constants. Code you don't write can't cause a segfault. FFI allows you to write less C, and from my experience the more C code you write the more likely you are to blow something up. FFI certainly doesn't protect you from other possible segfaults, like calling into libraries incorrectly or defining bad struct sizes or mismanaging memory, but it is at least less C code to write and maintain. I will grant there's a lot of up-front cost required (currently) that may make it no easier than maintaining all that C code. > With FFI you must: > > 1. Duplicate header files (see below for more problems) > 2. Understand struct layouts and the sizeof() for each member > 3. Do runtime checking of library features > 4. Worry about weak ref maps when using void pointers (see the id2ref > Â problem in nokogiri) > 5. Pay a runtime conversion price from ruby data types to FFI types > 6. Educate users on LD_LIBRARY_PATH > 7. Worry about 32bit and 64bit issues (like Tony mentioned) Yeah, I will admit there's more hassle using FFI than there should be. I don't know how to address that, but projects like ffi-inliner seem to be a step in the right direction. FFI-inliner basically allows you to have some embedded C code in your FFI-consuming library that it then compiles and links in via FFI. That allows you to get the compile-time tooling you want for wrangling nontrivial structs while still supporting any implementation that supports FFI. You lose the ability to run on platforms without a compiler available (though it does some wrangling with tcc, I believe), but it may be a good happy medium. What do you think? I don't want to give the impression that you shouldn't use C tooling to call a C library, or even that nobody should ever write C code. I just believe that everyone writing C code that depends on MRI's C API is a dead end. > Unfortunately, none of the problems I've just listed off are > theoretical. Â I have personally run in to every one of them and can > provide you with real world examples. Â FFI is awesome for certain, > confined, small, stable use cases. Â I use FFI, and I enjoy it. Â But > saying that it's "the only logical choice" seems wrong. I'll restate it: using mechanisms for binding C libraries that don't depend on MRI's C API is the only logical choice. FFI certainly isn't perfect, but it's the best option for doing that right now. > I am curious what your experience has been, and why you haven't run in to the > same problems? Â How do other people overcome these issues? We certainly have run into some of those issues, most notably when trying to support "stat" calls from JRuby across all platforms. Our only option has been to rewire the struct and call for each platform we intend to run on. It sucks, I agree. But we support stat on all those platforms out of a single JRuby distribution without a recompile being necessary. That's pretty cool. - Charlie |