Subject:
[ruby-ffi] Re: using :buffer_out with strings in ruby vs jruby
From:
Wayne Meissner
Date:
9/14/09 7:00 PM
To:
ruby-ffi@googlegroups.com


They're actually handled in the same way.

Both Buffer and string in JRuby are backed by a java byte[] array, and
to pass that to a native function, a temporary memory area is
allocated, the contents are copied in to the native memory, then the
temporary native memory is passed to the function.  After the function
call, the native memory is copied back to the byte[] array (if
requested), and the native memory is freed before returning back to
java/ruby.

Thats pretty much the difference between Buffer and MemoryPointer.
With MemoryPointer, you are allocating the native memory immediately
when you create the MemoryPointer, and it gets freed when
MemoryPointer gets garbage collected.  You get complete control over
the memory life cycle, so you can pass a MemoryPointer in to a
function that will keep a reference to the address.

With Buffer, the native memory is considered to be transient - this
allows the implementation (like JRuby) to optimize when/how native
memory is actually allocated/released for it, which can make a huge
difference (10x or more) when used as e.g. passing an int
by-reference.

2009/9/15 Aman Gupta <themastermind1@gmail.com>:
>
> Thanks, changing it to :buffer_in works as expected.
>
> When I create an explicit Buffer and pass it in using :pointer, the
> memory allocated for that buffer is freed when the ruby object is
> garbage collected. What happens when I pass in a string as :buffer_in
> instead? The contents of the string are copied into a temporary
> buffer, but when is the buffer freed? or does it live on the stack?
>
>  Aman
>
> On Mon, Sep 14, 2009 at 7:25 AM, Wayne Meissner <wmeissner@gmail.com> wrote:
>>
>> If that code is doing what I think it is doing, you have the direction wrong.
>>
>> :buffer_out == copy out from native memory to ruby memory
>> :buffer_in == copy in to native memory from ruby memory
>>
>> Think of them like IN and OUT parameter specifications.  IN means the
>> data is passed IN to the function, OUT means the function can alter
>> the value of the parameter.  INOUT means both.
>>
>> As to why the tests fail if you pass the string first, its because the
>> string gets overwritten with garbage from native memory due to the OUT
>> param spec.
>>
>> The tests work with :pointer, because it is basically the same as
>> :buffer_inout when it comes to string and Buffer arguments - the data
>> is first copied IN to native memory from ruby, then after the call,
>> the native data is copied OUT from native memory to ruby,  overwriting
>> the data with the same thing as was already there.
>>
>> 2009/9/14 Aman Gupta <themastermind1@gmail.com>:
>>>
>>> I'm seeing different behavior on ruby vs jruby when passing in a
>>> string as a :buffer_out argument:
>>>
>>> require 'rubygems'
>>> require 'ffi'
>>>
>>> module GLib
>>>  extend FFI::Library
>>>  ffi_lib 'libglib-2.0'
>>>  attach_function :g_memdup_buf, :g_memdup,
>>> [ :buffer_out, :uint ], :pointer
>>>  attach_function :g_memdup_ptr, :g_memdup,
>>> [ :pointer, :uint ], :pointer
>>>
>>>  def self.test_memdup(arg_type, func_type)
>>>    ostr = "a\0b\0c"
>>>
>>>    if arg_type == :string
>>>      arg = ostr.dup
>>>    elsif arg_type == :buffer
>>>      arg = FFI::Buffer.new(ostr.size)
>>>      arg.put_bytes(0, ostr)
>>>    end
>>>
>>>    if func_type == :buffer_out
>>>      ret = GLib.g_memdup_buf(arg, arg.size)
>>>    elsif func_type == :pointer
>>>      ret = GLib.g_memdup_ptr(arg, arg.size)
>>>    end
>>>
>>>    rstr = ret.get_bytes(0, ostr.size)
>>>
>>>    p [arg_type, func_type, ostr, rstr, ostr == rstr]
>>>  end
>>> end
>>>
>>> GLib.test_memdup(:buffer, :buffer_out)
>>> GLib.test_memdup(:buffer, :pointer)
>>> GLib.test_memdup(:string, :buffer_out)
>>> GLib.test_memdup(:string, :pointer)
>>>
>>> $ ruby glib_memduptest.rb
>>> [:buffer, :buffer_out, "a\000b\000c", "a\000b\000c", true]
>>> [:buffer, :pointer, "a\000b\000c", "a\000b\000c", true]
>>> [:string, :buffer_out, "a\000b\000c", "a\000b\000c", true]
>>> [:string, :pointer, "a\000b\000c", "a\000b\000c", true]
>>>
>>> $ jruby glib_memduptest.rb
>>> [:buffer, :buffer_out, "a\000b\000c", "\300\370\177\260A", false]
>>> [:buffer, :pointer, "a\000b\000c", "a\000b\000c", true]
>>> [:string, :buffer_out, "\001\000\000\000\270", "\001\000\000\000\270",
>>> true]
>>> [:string, :pointer, "\001\000\000\000\270", "\001\000\000\000\270",
>>> true]
>>>
>>> I'm not really sure what's happening in the jruby case. If I use an
>>> explicit FFI::Buffer with :pointer, it seems to work.
>>>
>>> What's really strange though, is that if you reverse the order in
>>> which the tests happen (so the string versions execute first), they
>>> all fail:
>>>
>>> GLib.test_memdup(:string, :buffer_out)
>>> GLib.test_memdup(:string, :pointer)
>>> GLib.test_memdup(:buffer, :buffer_out)
>>> GLib.test_memdup(:buffer, :pointer)
>>>
>>> $ jruby glib_memduptest.rb
>>> [:string, :buffer_out, "\300\370\177\260A", "\300\370\177\260A", true]
>>> [:string, :pointer, "\300\370\177\260A", "\300\370\177\260A", true]
>>> [:buffer, :buffer_out, "\300\370\177\260A", "0\f\200\001@", false]
>>> [:buffer, :pointer, "\300\370\177\260A", "\300\370\177\260A", true]
>>>
>>
>