Python API about the Shared Memory Transport

[cdp-study]

I want to use Fast-DDS-Python transfers the 900K data, but I find it too slow to transfer data. By viewing the answers to the corresponding questions in Fast-DDS, I need to use the Shared Memory Transport. But I don't find it in the Python API. Can you tell me how to implement the corresponding functions with the Fast-DDS-Python?

[duchengyao]

Did you find a solution >_<

[jsan-rt]

Hi @cdp-study , @duchengyao ,

At this moment it's not possible to define the user transports using the Python API. However, by default Fast DDS uses both an UDPv4 transport and a Shared Memory transport, the latter being preferred for data transfer if communication is possible.

Could you provide us with some more information (like the IDL used to generate the type) or a minimal reproducer?

[duchengyao]

@jsantiago-eProsima Thanks for reply.

For example, I'd like to receive data from example/HelloWorldExampleSharedMem .

1. I modified HelloWorld.idl #L5 to char data[1048576]; because of this issue
2. Generated and build using fastddsgen -python HelloWorld.idl mkdir build; cmake ..; make
3. Copied HelloWorldExample.py to build folder. And modified the topic name to "HelloWorldSharedMemTopic", "HelloWorld".

Then, I run publisher of C++ and subscriber of python. The subscriber cannot receive any data.

But after delete this line pqos.transport().use_builtin_transports = false; #L57 ,the subscriber could receive data.

So, maybe the python binding cannot receive data of shared memory.
Is there the way to receive data of HelloWorldExampleSharedMem without modify the code of C++?

Thanks!

[jsan-rt]

You could use an XML profile like the one shown here to setup the available descriptors for your participant. This XML can be loaded either from a file or from a string.

If the profile is not set as default, you can specify it manually on participant creation with the create_participant_with_profile call.

[duchengyao]

@jsantiago-eProsima I have successfully used shared memory via XML, thanks!

BTW,
Converting SWIG type to numpy or list will consume a lot of time. I can't find the solution about that, could you please provide an example, thank you ^_^

python:

data.raw_data()
>>    <...;proxy of <Swig Object of type 'std::array< float,512*16000 > *' at 0x7fb...> >

a = list(data.raw_data())        # cost long time
b = np.array(data.raw_data())    # cost long time

And I try using ctypes to accelerate speed of converting data.

a = np.frombuffer((ctypes.c_float*512*16000).from_address(0x7fb..),np.float32)

But I cannot get the address using variable. Can I get the address like data.raw_data().get_address() ?

[JesusPoderoso]

Hi @duchengyao, sorry for the late response. The data conversion has been improved in this PR, which introduces a get_buffer() method that returns the data buffer. The time cost reduction is significant, but its usage is quite similar to the solution you reached:

a = np.frombuffer((ctypes.c_float*512*16000).from_address(int(data.get_buffer())),np.float32)

But I cannot get the address using variable. Can I get the address like data.raw_data().get_address() ?

Regarding this, the get_buffer() method can be used on the defined types: data.address().get_buffer().

[JesusPoderoso]

I am moving this issue to the support discussion forum as per our CONTRIBUTING guidelines.