Subject:
Re: [ruby-ffi] Struct/ManagedStruct and GC behaviour
From:
Jon
Date:
12/23/09 11:48 AM
To:
ruby-ffi@googlegroups.com

> On Mon, Dec 21, 2009 at 4:20 PM, Wayne Meissner <wmeissner@gmail.com> wrote:
> > 2009/12/22 Thomas E Enebo <tom.enebo@gmail.com>:
> >> I have been wondering about the Struct API myself.   It seems like it
> >> should support a simple explicit memory management API ala:
> >>
> >> class MyFFIStruct < FFI::Struct ....
> >>
> >> MyFFIStruct.new do |struct_instance|
> >>   some_ffi_call struct_instance
> >>   # Do other stuff....
> >> end
> >> # I know the block has disposed of my allocated struct
> >
> > It does support this indirectly via MemoryPointer.new
> > e.g.
> >  MemoryPointer.new(MyFFIStruct.size) do |p|
> >    s = MyFFIStruct.new(p)
> >    some_ffi_call s
> >    # do other stuff
> >  end
> >
> > An alternative is to use FFI::Buffer as the backing memory - on JRuby,
> > that is jvm-heap memory, which is copied in/out of a temporary native
> > memory area (usually on the stack) when passed to a native function.
> > For small structs, especially ones that are passed to a native
> > function just once, this can be up to 10x faster than the alloc/free
> > cycle of a MemoryPointer backed Struct.
> 
> Thanks for the additional info and the code snippet for explicit
> management.  Personally, I think that an explicit API like this should
> be the norm since there are no guarantees on when GC happens and in my
> experience it is much longer than most people expect.   I think too
> many people believe in magic and the only way to combat it is to make
> the implicit stuff be more typing (or the explicit stuff less typing).
>   I will probably make a class method in Struct which allows something
> like:
> 
> MyFFStruct.create do |struct|
> ...
> end
> 
> Under the covers it will do what you demonstrated above:
> 
> def self.create(&block)
>     MemoryPointer.new(size) { |p| block.call new(p) }
> end
> 
> Or something like this...
> 
> -Tom


As another API-level option to Tom's, where's the holes this wild-eyed-unresearched-NOT-a-proposal brainstorming idea?

module FFI
  def self.autofree(instance, free_proc=nil, options={}, &block)
    # do your stuff in your block
    # if your instance has a 'free', i'll call it (ruby specific cleanup)
    # don't give me a 'free_proc' and i'll call my FFI::DEFAULT_FREE Proc to free your native resources (!?)
    # give me a 'free_proc' and i'll call it to free your native resources
  end

typical usage:

  autofree MyFFIStruct.new do |instance|
    ...
  end


* explicit
* applicable to any instance (!!??)
* similar usage to Python's 'with'[1] and C#'s 'using' statements
* handle 80% of common usage cases via the FFI::DEFAULT_FREE Proc while providing an escape hatch to address Evan's comments and Wayne's "no-win" comment (does a custom 'free_proc' Proc open up the door to a bunch of circular dependency issues?)
* abstracts impl-specific stuff to address Wayne's "in hindsight [refactoring]" comments (potential for reusing/repacking much of the existing battle-tested ffi code?)
* provides maximum flexibility to the different ffi implementations?


I still need to dig around in the nokogiri ffi code Wayne mentioned as well as the autopointer, etc code, but this little mind virus needs more & different perspectives than I'm able to give it :(  

Jon


[1] http://docs.python.org/reference/compound_stmts.html#the-with-statement