#7 Generated code by dbusxml2qt3 for type="a{oa{sa{sv}}}" does not compile

Open
opened 3 months ago by deloptes · 31 comments

Basic information

  • TDE version: R14.1
  • Distribution: Debian Stretch
  • Hardware: amd64/i386 perhaps all others

Description

When generated code for

        <interface name="org.freedesktop.DBus.ObjectManager">
            <method name="GetManagedObjects">
                <arg name="objects" type="a{oa{sa{sv}}}" direction="out" />
            </method>
        ...

the generated code does not compile. The error message is

objectmanagerinterface.cpp: In member function 'TQT_DBusMessage org::freedesktop::DBus::ObjectManager::callGetManagedObjects(const TQT_DBusMessage&)':
objectmanagerinterface.cpp:116:99: error: no matching function for call to 'TQT_DBusDataMap<TQT_DBusObjectPath>::TQT_DBusDataMap(TQMap<TQT_DBusObjectPath, TQT_DBusDataMap<TQString> >&)'
         reply << TQT_DBusData::fromObjectPathKeyMap(TQT_DBusDataMap< TQT_DBusObjectPath >(_objects));

Steps to reproduce

  1. Use the introspectable for ObjectManager from the exampl 4e https://mirror.git.trinitydesktop.org/gitea/deloptes/dbus-1-tqt-example/src/branch/master/4e/test1.xml

  2. Generate the code from test1.xml dbusxml2qt3 test1.xml

  3. Try to compile the example

Resolution

the interface

In the objectmanagerinterface.h file

 protected:
-    virtual bool GetManagedObjects(TQMap< TQT_DBusObjectPath, TQT_DBusDataMap< TQString > >& objects, TQT_DBusError& error) = 0;
+    virtual bool GetManagedObjects(TQT_DBusDataMap< TQT_DBusObjectPath >& objects, TQT_DBusError& error) = 0;

in the ./objectmanagerinterface.cpp file

-    TQMap< TQT_DBusObjectPath, TQT_DBusDataMap< TQString > > _objects;
+    TQT_DBusDataMap< TQT_DBusObjectPath > _objects;

the proxy

It is the same. As soon as “TQMap< TQT_DBusObjectPath, TQT_DBusDataMap< TQString > >” is replaced with “TQT_DBusDataMap< TQT_DBusObjectPath >& objects” all works fine

The question

The question is where this has to take place in dbusxml2qt3 or in dbus-1-tqt. I hope to find the answer soon.

<!-- This is a comment. Please fill in the required fields below. The comments provide instructions on how to do so. Note: You do not need to remove comments. --> ## Basic information - TDE version: R14.1 - Distribution: Debian Stretch - Hardware: amd64/i386 perhaps all others <!-- Use SL/* labels to set the severity level. Please do not set a milestone. --> ## Description When generated code for ``` <interface name="org.freedesktop.DBus.ObjectManager"> <method name="GetManagedObjects"> <arg name="objects" type="a{oa{sa{sv}}}" direction="out" /> </method> ... ``` the generated code does not compile. The error message is ``` objectmanagerinterface.cpp: In member function 'TQT_DBusMessage org::freedesktop::DBus::ObjectManager::callGetManagedObjects(const TQT_DBusMessage&)': objectmanagerinterface.cpp:116:99: error: no matching function for call to 'TQT_DBusDataMap<TQT_DBusObjectPath>::TQT_DBusDataMap(TQMap<TQT_DBusObjectPath, TQT_DBusDataMap<TQString> >&)' reply << TQT_DBusData::fromObjectPathKeyMap(TQT_DBusDataMap< TQT_DBusObjectPath >(_objects)); ``` ## Steps to reproduce 1. Use the introspectable for ObjectManager from the exampl 4e https://mirror.git.trinitydesktop.org/gitea/deloptes/dbus-1-tqt-example/src/branch/master/4e/test1.xml 2. Generate the code from test1.xml ```dbusxml2qt3 test1.xml``` 3. Try to compile the example ## Resolution ### the interface In the objectmanagerinterface.h file ``` protected: - virtual bool GetManagedObjects(TQMap< TQT_DBusObjectPath, TQT_DBusDataMap< TQString > >& objects, TQT_DBusError& error) = 0; + virtual bool GetManagedObjects(TQT_DBusDataMap< TQT_DBusObjectPath >& objects, TQT_DBusError& error) = 0; ``` in the ./objectmanagerinterface.cpp file ``` - TQMap< TQT_DBusObjectPath, TQT_DBusDataMap< TQString > > _objects; + TQT_DBusDataMap< TQT_DBusObjectPath > _objects; ``` ### the proxy It is the same. As soon as "TQMap< TQT_DBusObjectPath, TQT_DBusDataMap< TQString > >" is replaced with "TQT_DBusDataMap< TQT_DBusObjectPath >& objects" all works fine ## The question The question is where this has to take place in dbusxml2qt3 or in dbus-1-tqt. I hope to find the answer soon.
deloptes commented 3 months ago
Poster

sorry closed by mistake

sorry closed by mistake
MicheleC commented 3 months ago
Owner

Thanks Emanoil, I will take a look at this after I am done with bug 1923. I had some problems in the past to get TQT_DBusDataMap to work but ultimately I was able to compile what I wanted, so it may be a similar issue here, perhaps.

Thanks Emanoil, I will take a look at this after I am done with bug 1923. I had some problems in the past to get TQT_DBusDataMap to work but ultimately I was able to compile what I wanted, so it may be a similar issue here, perhaps.
deloptes commented 3 months ago
Poster

No problem take your time. I guess at the moment the code was written there must have been something with ObjectPath arrays - perhaps it was not existing? Recall the other bug in bugzilla, that dbus-1-tqt was not handling “oa” at all or as string array - not exactly sure anymore.

I also generated the skeleton code for TDEs hardwarecontrol, but transferring the functionality from the C code is a bit hard exercise :)

No problem take your time. I guess at the moment the code was written there must have been something with ObjectPath arrays - perhaps it was not existing? Recall the other bug in bugzilla, that dbus-1-tqt was not handling "oa" at all or as string array - not exactly sure anymore. I also generated the skeleton code for TDEs hardwarecontrol, but transferring the functionality from the C code is a bit hard exercise :)
MicheleC commented 2 months ago
Owner

Just for info, I have started to look into this.

Just for info, I have started to look into this.
MicheleC commented 2 months ago
Owner

I have taken a detailed look at this issue and associated PR #8 and I think the solution proposed in #8 is not the best way.

There is nothing wrong in the method generated by dbusxml2qt3 for this signature “a{oa{sa{sv}}}”. It creates a TQMap where the key is of type TQT_DBusObjectPath and the values are of type TQT_DBusDataMap. The problem is that there is no constructor for TQT_DBusDataMap from a TQMap>, so trying to compile the generated code for such case fails.

    TQMap< TQT_DBusObjectPath, TQT_DBusDataMap< TQString > > _objects;
    if (GetManagedObjects(_objects, error))
    {
        reply = TQT_DBusMessage::methodReply(message);
        reply << TQT_DBusData::fromObjectPathKeyMap(TQT_DBusDataMap< TQT_DBusObjectPath >(_objects));

The solution proposed in #8 fixes the compilation problem but the generated object is a more generic TQT_DBusDataMap, meaning that more work will be required when we want to use such object.
In my opinion it would be better to add a

    TQT_DBusDataMap<T>(const TQMap<T, TQT_DBusDataMap<U>>& other)
        : TQMap<T, TQT_DBusData>(), m_valueType(TQT_DBusData::Invalid)

constructor for TQT_DBusDataMap.
What do you think?

I have taken a detailed look at this issue and associated PR #8 and I think the solution proposed in #8 is not the best way. There is nothing wrong in the method generated by dbusxml2qt3 for this signature "a{oa{sa{sv}}}". It creates a TQMap where the key is of type TQT_DBusObjectPath and the values are of type TQT_DBusDataMap<TQString>. The problem is that there is no constructor for TQT_DBusDataMap from a TQMap<T, TQT_DBusDataMap<U>>, so trying to compile the generated code for such case fails. ``` TQMap< TQT_DBusObjectPath, TQT_DBusDataMap< TQString > > _objects; if (GetManagedObjects(_objects, error)) { reply = TQT_DBusMessage::methodReply(message); reply << TQT_DBusData::fromObjectPathKeyMap(TQT_DBusDataMap< TQT_DBusObjectPath >(_objects)); ``` The solution proposed in #8 fixes the compilation problem but the generated object is a more generic TQT_DBusDataMap<TQT_DBusObjectPath>, meaning that more work will be required when we want to use such object.<br> In my opinion it would be better to add a ``` TQT_DBusDataMap<T>(const TQMap<T, TQT_DBusDataMap<U>>& other) : TQMap<T, TQT_DBusData>(), m_valueType(TQT_DBusData::Invalid) ``` constructor for TQT_DBusDataMap.<br> What do you think?
MicheleC commented 2 months ago
Owner

PR #18 propose a new way to fix this problem, by adding the methods mentioned in the previous comments.

Referring to #8, PR #18 is meant to replace commit 2272ebde20. For commit 5198e048e1 I will add further comments after we settle the above point.

PR #18 propose a new way to fix this problem, by adding the methods mentioned in the previous comments. Referring to #8, PR #18 is meant to replace commit 2272ebde20. For commit 5198e048e1 I will add further comments after we settle the above point.
deloptes commented 1 month ago
Poster

Hi, I backed off touching dbus-1-tqt, but I was also thinking of what you proposed and that probably this would be “the proper” solution, however reusing what dbus-1-tqt offers in this simple way was too tempting :).

I will be looking at and testing with #18 end of the week hopefully.

Hi, I backed off touching dbus-1-tqt, but I was also thinking of what you proposed and that probably this would be "the proper" solution, however reusing what dbus-1-tqt offers in this simple way was too tempting :). I will be looking at and testing with #18 end of the week hopefully.
MicheleC commented 1 month ago
Owner

Yeah, I see what you mean by “tempting” :smile: nevertheless the problem was only delayed till you had to actually use the returned map. PR #18 does not alter the signature of generated methods and extends the capability to build TQT_DBusDataMap from more complex objects, therefore it should be less risky and more powerful.

Take your time, I will be waiting for your feedback on #18. With that, I am able to successfully build your examples 4d and 4e (minor changes required to use correct include file names and to update the call in Interface2 to match the unchanged object manager interface method). I have not tried out 4f since this is on a later PR.

Yeah, I see what you mean by "tempting" :smile: nevertheless the problem was only delayed till you had to actually use the returned map. PR #18 does not alter the signature of generated methods and extends the capability to build TQT_DBusDataMap from more complex objects, therefore it should be less risky and more powerful. Take your time, I will be waiting for your feedback on #18. With that, I am able to successfully build your examples 4d and 4e (minor changes required to use correct include file names and to update the call in Interface2 to match the unchanged object manager interface method). I have not tried out 4f since this is on a later PR.
deloptes commented 1 month ago
Poster

Hi Michele,

your commit 00b00f9961

Author: Michele Calgaro <michele.calgaro@yahoo.it>
Date:   Mon May 20 23:22:18 2019 +0900

Modified TQT_DBusObjectPath to inherit from TQString instead of
TQCString.

Signed-off-by: Michele Calgaro <michele.calgaro@yahoo.it>

breaks a lot in TDE

 /mnt/DEVELOPMENT/TDE/repo-master/tde/2_build/tdelibs/tdecore/tdehw/networkbackends/network-manager/network-manager.cpp:4607:54: error: 'class TQT_DBusObjectPath' has  no member named 'data'; did you mean 'at'?
    if (!d->nmAddConnectionAsyncResponse[asyncCallID].data()) {
                                                      ^~~~
 /mnt/DEVELOPMENT/TDE/repo-master/tde/2_build/tdelibs/tdecore/tdehw/networkbackends/network-manager/network-manager.cpp: In member function 'TQCString TDENetworkConnectionManager_BackendNM::getActiveConnectionPath(TQString)':


 /usr/include/dbus-1-tqt/tqdbusobjectpath.h:38:21: note: candidate: TQT_DBusObjectPath& TQT_DBusObjectPath::operator=(const TQT_DBusObjectPath&)
 class TQDBUS_EXPORT TQT_DBusObjectPath : public TQString
                     ^~~~~~~~~~~~~~~~~~

can we fix this first before we proceed

Hi Michele, your commit 00b00f996127cbb0e9d677494494b971cbdadb3d Author: Michele Calgaro <michele.calgaro@yahoo.it> Date: Mon May 20 23:22:18 2019 +0900 Modified TQT_DBusObjectPath to inherit from TQString instead of TQCString. Signed-off-by: Michele Calgaro <michele.calgaro@yahoo.it> breaks a lot in TDE ``` /mnt/DEVELOPMENT/TDE/repo-master/tde/2_build/tdelibs/tdecore/tdehw/networkbackends/network-manager/network-manager.cpp:4607:54: error: 'class TQT_DBusObjectPath' has no member named 'data'; did you mean 'at'? if (!d->nmAddConnectionAsyncResponse[asyncCallID].data()) { ^~~~ /mnt/DEVELOPMENT/TDE/repo-master/tde/2_build/tdelibs/tdecore/tdehw/networkbackends/network-manager/network-manager.cpp: In member function 'TQCString TDENetworkConnectionManager_BackendNM::getActiveConnectionPath(TQString)': /usr/include/dbus-1-tqt/tqdbusobjectpath.h:38:21: note: candidate: TQT_DBusObjectPath& TQT_DBusObjectPath::operator=(const TQT_DBusObjectPath&) class TQDBUS_EXPORT TQT_DBusObjectPath : public TQString ^~~~~~~~~~~~~~~~~~ ``` can we fix this first before we proceed
SlavekB commented 1 month ago
Owner

It’s possible that you missed the commit 080948356f in tdelibs?

In addition to the commit for tdelibs, you need to rebuild tdebase and tdepowersave.

It's possible that you missed the commit [080948356f](../tdelibs/commit/080948356f) in tdelibs? In addition to the commit for tdelibs, you need to rebuild tdebase and tdepowersave.
MicheleC commented 1 month ago
Owner

Hi Emanoil, yes, the change in dbus-1-tqt caused FTBFS in tdelibs. Nevertheless as Slavek already pointed out, I have already pushed a fix for that.
Rebuilding of tdebase and tdepowersave is necessary to avoid crashes after the next reboot.

Hi Emanoil, yes, the change in dbus-1-tqt caused FTBFS in tdelibs. Nevertheless as Slavek already pointed out, I have already pushed a fix for that.<br> Rebuilding of tdebase and tdepowersave is necessary to avoid crashes after the next reboot.<br>
deloptes commented 1 month ago
Poster

Thanks, for some reason I have problem updating the repository. I first tried the script then “repo-master/tde/1_git/tde$ git pull --recurse-submodules”.

Only when I changed in tdelibs it did update.

I now run again “repo-master/tde/1_git/tde$ git pull --recurse-submodules” and it pulls updates

Fetching submodule main/tde-i18n
From https://scm.trinitydesktop.org/scm/git/tde-i18n
   9ab84adea6..1ba25ef021  master     -> origin/master
   f8d90fd2e6..5929d507da  r14.0.x    -> origin/r14.0.x

But it is not pulling from TGW, but from scm.

Thanks, for some reason I have problem updating the repository. I first tried the script then "repo-master/tde/1_git/tde$ git pull --recurse-submodules". Only when I changed in tdelibs it did update. I now run again "repo-master/tde/1_git/tde$ git pull --recurse-submodules" and it pulls updates ``` Fetching submodule main/tde-i18n From https://scm.trinitydesktop.org/scm/git/tde-i18n 9ab84adea6..1ba25ef021 master -> origin/master f8d90fd2e6..5929d507da r14.0.x -> origin/r14.0.x ``` But it is not pulling from TGW, but from scm.
MicheleC commented 1 month ago
Owner

you need to change the path of your origin if you want to pull from TGW.

git remote rm origin
git remote add origin <path>
you need to change the path of your origin if you want to pull from TGW. ``` git remote rm origin git remote add origin <path> ```
SlavekB commented 1 month ago
Owner

Instead of removing / adding, you can use the command:

git remote set-url origin <new-url>
Instead of removing / adding, you can use the command: ``` git remote set-url origin <new-url> ```
MicheleC commented 1 month ago
Owner

thanks Slavek, learnt something new today :+1:

thanks Slavek, learnt something new today :+1:
deloptes commented 1 month ago
Poster

As mentioned by you. I had to adjust the files testservice.* in example 4e and the build script to match the new signature and node, interface, proxy with caps.

I also had to update rootnode.* to support bool registerObject(const TQT_DBusConnection& connection, const TQString& path, const TQString& next); and now it builds and works properly.

Regarding this registerObject and the part that belongs to generated rootNode, I guess we’ll handle it in another round, so it means so far last patch to dbus-1-tqt is good indeed :+1: .

As mentioned by you. I had to adjust the files testservice.* in example 4e and the build script to match the new signature and node, interface, proxy with caps. I also had to update rootnode.* to support `bool registerObject(const TQT_DBusConnection& connection, const TQString& path, const TQString& next);` and now it builds and works properly. Regarding this registerObject and the part that belongs to generated rootNode, I guess we'll handle it in another round, so it means so far last patch to dbus-1-tqt is good indeed :+1: .
MicheleC commented 1 month ago
Owner

Ok, thanks for the feedback Emainoil. I will proceed and merge the patch. Then sometime next week I will look at the generated node and comment accordingly.

PS: previous comment (now deleted) was meant for another PR. Got confused :confused:

Ok, thanks for the feedback Emainoil. I will proceed and merge the patch. Then sometime next week I will look at the generated node and comment accordingly. PS: previous comment (now deleted) was meant for another PR. Got confused :confused:
deloptes commented 1 month ago
Poster

OT: In fact changing the submodule path turned out to be much more complicated than anticipated. The command git remote set-url origin <new-url> changes only the path of the repository and not of the submodule. As you have many submodules under submodules it is not clear how one should replace all the urls. Different ideas all around the net - nothing working 100%.

The most efficient way I found was (for example admin and cmake):

git config submodule.cmake.url https://mirror.git.trinitydesktop.org/gitea/TDE/tde-common-cmake
git config submodule.admin.url https://mirror.git.trinitydesktop.org/gitea/TDE/tde-common-admin

forgot to mention you need to execute git submodule sync after this to get the record actually from .gitmodules into .git/config

OT: In fact changing the submodule path turned out to be much more complicated than anticipated. The command `git remote set-url origin <new-url>` changes only the path of the repository and not of the submodule. As you have many submodules under submodules it is not clear how one should replace all the urls. Different ideas all around the net - nothing working 100%. The most efficient way I found was (for example admin and cmake): ``` git config submodule.cmake.url https://mirror.git.trinitydesktop.org/gitea/TDE/tde-common-cmake git config submodule.admin.url https://mirror.git.trinitydesktop.org/gitea/TDE/tde-common-admin ``` forgot to mention you need to execute `git submodule sync` after this to get the record actually from .gitmodules into .git/config
SlavekB commented 1 month ago
Owner

Yes, a change in all submodules can be challenging. You can try to use for example:

git submodule foreach --recursive "git config remote.origin.url | sed 's|http://scm.trinitydesktop.org/scm/git/|https://mirror.git.trinitydesktop.org/gitea/TDE/|' | xargs git remote set-url origin"
Yes, a change in all submodules can be challenging. You can try to use for example: ``` git submodule foreach --recursive "git config remote.origin.url | sed 's|http://scm.trinitydesktop.org/scm/git/|https://mirror.git.trinitydesktop.org/gitea/TDE/|' | xargs git remote set-url origin" ```
deloptes commented 1 month ago
Poster

Thanks Slavek, I did something similar with perl, but I have to update another repository I keep here, so I will try your suggestion.

Thanks!

Thanks Slavek, I did something similar with perl, but I have to update another repository I keep here, so I will try your suggestion. Thanks!
MicheleC commented 1 month ago
Owner

Hi Emanoil,
finally I come to comment on the node generation code. With reference to example 4e from your private repository, I find the code that creates all node levels a bit repetitive and it is not clear to me why we need to do so. For example in main.cpp you have:

RootNodeService rootService(connection);
  OrgNodeService orgService(connection);
  ExampleNodeService exampleService(connection);
  MultiInterfaceService service(connection);

then each node service is basically identical to the others except for the registered path. Is this really necessary and if so, why? By the way, using the example 4e from your repo and try to accessing it from d-feet, it seems somethign is wrong: I see no interface available in the servie and there a circle that keeps running around forever near the “refresh” button at the top right corner.
To proceed further, could you please provide feedback on the following points? Thanks.

  1. why do you think it is necessary to register a node for each level? What benefit does it brings? This is not clear to me yet
  2. can you fix up the code in your repo to adapt to latest dbus-1-tqt code (if necessary)
  3. can you look at 4e and make sure it works fine with d-feet?
Hi Emanoil,<br> finally I come to comment on the node generation code. With reference to example 4e from your private repository, I find the code that creates all node levels a bit repetitive and it is not clear to me why we need to do so. For example in main.cpp you have: ``` RootNodeService rootService(connection); OrgNodeService orgService(connection); ExampleNodeService exampleService(connection); MultiInterfaceService service(connection); ``` then each node service is basically identical to the others except for the registered path. Is this really necessary and if so, why? By the way, using the example 4e from your repo and try to accessing it from d-feet, it seems somethign is wrong: I see no interface available in the servie and there a circle that keeps running around forever near the "refresh" button at the top right corner.<br> To proceed further, could you please provide feedback on the following points? Thanks. 1. why do you think it is necessary to register a node for each level? What benefit does it brings? This is not clear to me yet 2. can you fix up the code in your repo to adapt to latest dbus-1-tqt code (if necessary) 3. can you look at 4e and make sure it works fine with d-feet?
deloptes commented 1 month ago
Poster

Hi Michele, when experimenting I found this is the easiest way to do the trick.

The requirement is that each node returns introspection of itself plus a node element of the child (or children), but let keep it simple with one child.

Again to keep it simple I did not find a way to parse the path in the code. This requires a lot of modifications of the generated code, so using the one node class with that small fix solves the problem if it is registered at each node level holding its child (children).

The purpose of those RootNodeService, OrgNodeService and ExampleNodeService are to return the content of the node.

Second purpose is to be able to register any kind of service. Look how registering is done. This gives you freedom to register a node that holds service and child node or node without services but with child node etc. which solves the problem described in issue #3.

I’ll update the example based on current code as requested in 2. If there are problems I’ll post here.

Regarding 3) d-feet - first time I hear of this tool. I tend to avoid python. No idea why it does not work. I’ll need to have a look - but it is just an example code. ObjectManager is not implemented fully (see the reply), so may be it has to do with that. I tested the example with dbus-send (see below) and qdbusviewer (in the screenshots). Perhaps I’ll replace ObjectManager with something else to not confuse the audience. I took it as typical example of the use case.

I’ll give you a note when 2 and 3 are done - it might take a while.

dbus-send --print-reply --session --dest=org.example.Service /    org.freedesktop.DBus.Introspectable.Introspect

<!DOCTYPE node PUBLIC "-//freedesktop//DTD D-BUS Object Introspection 1.0//EN"
"http://www.freedesktop.org/standards/dbus/1.0/introspect.dtd">
<node>
 <interface name="org.freedesktop.DBus.Introspectable" >
  <method name="Introspect" >
   <arg direction="out" type="s" name="xml" />
  </method>
 </interface>
 <interface name="org.freedesktop.DBus.ObjectManager" >
  <method name="GetManagedObjects" >
   <arg direction="out" type="a{oa{sa{sv}}}" name="objects" />
  </method>
  <signal name="InterfacesAdded" >
   <arg direction="in" type="o" name="object" />
   <arg direction="in" type="a{sa{sv}}" name="interfaces" />
  </signal>
  <signal name="InterfacesRemoved" >
   <arg direction="in" type="o" name="object" />
   <arg direction="in" type="as" name="interfaces" />
  </signal>
 </interface>
 <node name="org" />
</node>

dbus-send --print-reply --session --dest=org.example.Service /org    org.freedesktop.DBus.Introspectable.Introspect

<!DOCTYPE node PUBLIC "-//freedesktop//DTD D-BUS Object Introspection 1.0//EN"
"http://www.freedesktop.org/standards/dbus/1.0/introspect.dtd">
<node name="/org" >
 <node name="example" />
</node>

dbus-send --print-reply --session --dest=org.example.Service /org/example    org.freedesktop.DBus.Introspectable.Introspect

<!DOCTYPE node PUBLIC "-//freedesktop//DTD D-BUS Object Introspection 1.0//EN"
"http://www.freedesktop.org/standards/dbus/1.0/introspect.dtd">
<node name="/org/example" >
 <node name="service" />
</node>

dbus-send --print-reply --session --dest=org.example.Service /org/example/service    org.freedesktop.DBus.Introspectable.Introspect
<!DOCTYPE node PUBLIC "-//freedesktop//DTD D-BUS Object Introspection 1.0//EN"
"http://www.freedesktop.org/standards/dbus/1.0/introspect.dtd">
<node name="/org/example/service" >
 <interface name="org.freedesktop.DBus.Introspectable" >
  <method name="Introspect" >
   <arg direction="out" type="s" name="xml" />
  </method>
 </interface>
 <interface name="org.example.Service" >
  <method name="ListSorter" >
   <arg direction="in" type="as" name="input" />
   <arg direction="out" type="as" name="output" />
  </method>
 </interface>
</node>
Hi Michele, when experimenting I found this is the easiest way to do the trick. The requirement is that each node returns introspection of itself plus a node element of the child (or children), but let keep it simple with one child. Again to keep it simple I did not find a way to parse the path in the code. This requires a lot of modifications of the generated code, so using the one node class with that small fix solves the problem if it is registered at each node level holding its child (children). The purpose of those RootNodeService, OrgNodeService and ExampleNodeService are to return the content of the node. Second purpose is to be able to register any kind of service. Look how registering is done. This gives you freedom to register a node that holds service and child node or node without services but with child node etc. which solves the problem described in issue #3. I'll update the example based on current code as requested in 2. If there are problems I'll post here. Regarding 3) d-feet - first time I hear of this tool. I tend to avoid python. No idea why it does not work. I'll need to have a look - but it is just an example code. ObjectManager is not implemented fully (see the reply), so may be it has to do with that. I tested the example with dbus-send (see below) and qdbusviewer (in the screenshots). Perhaps I'll replace ObjectManager with something else to not confuse the audience. I took it as typical example of the use case. I'll give you a note when 2 and 3 are done - it might take a while. dbus-send --print-reply --session --dest=org.example.Service / org.freedesktop.DBus.Introspectable.Introspect <!DOCTYPE node PUBLIC "-//freedesktop//DTD D-BUS Object Introspection 1.0//EN" "http://www.freedesktop.org/standards/dbus/1.0/introspect.dtd"> <node> <interface name="org.freedesktop.DBus.Introspectable" > <method name="Introspect" > <arg direction="out" type="s" name="xml" /> </method> </interface> <interface name="org.freedesktop.DBus.ObjectManager" > <method name="GetManagedObjects" > <arg direction="out" type="a{oa{sa{sv}}}" name="objects" /> </method> <signal name="InterfacesAdded" > <arg direction="in" type="o" name="object" /> <arg direction="in" type="a{sa{sv}}" name="interfaces" /> </signal> <signal name="InterfacesRemoved" > <arg direction="in" type="o" name="object" /> <arg direction="in" type="as" name="interfaces" /> </signal> </interface> <node name="org" /> </node> dbus-send --print-reply --session --dest=org.example.Service /org org.freedesktop.DBus.Introspectable.Introspect <!DOCTYPE node PUBLIC "-//freedesktop//DTD D-BUS Object Introspection 1.0//EN" "http://www.freedesktop.org/standards/dbus/1.0/introspect.dtd"> <node name="/org" > <node name="example" /> </node> dbus-send --print-reply --session --dest=org.example.Service /org/example org.freedesktop.DBus.Introspectable.Introspect <!DOCTYPE node PUBLIC "-//freedesktop//DTD D-BUS Object Introspection 1.0//EN" "http://www.freedesktop.org/standards/dbus/1.0/introspect.dtd"> <node name="/org/example" > <node name="service" /> </node> dbus-send --print-reply --session --dest=org.example.Service /org/example/service org.freedesktop.DBus.Introspectable.Introspect <!DOCTYPE node PUBLIC "-//freedesktop//DTD D-BUS Object Introspection 1.0//EN" "http://www.freedesktop.org/standards/dbus/1.0/introspect.dtd"> <node name="/org/example/service" > <interface name="org.freedesktop.DBus.Introspectable" > <method name="Introspect" > <arg direction="out" type="s" name="xml" /> </method> </interface> <interface name="org.example.Service" > <method name="ListSorter" > <arg direction="in" type="as" name="input" /> <arg direction="out" type="as" name="output" /> </method> </interface> </node>
MicheleC commented 1 month ago
Owner

Hi Emanoil, thanks for the feedback. Here is my view on the various points.

  1. registering each level of a path I think this is conceptually wrong as I mentioned in previous comments. Names on the bus have to be unique for dbus to work correctly. Now imagine two different applications providing two different services and interface:
    /org/example/app1/interface1
    /org/example/app2/interface2
    If those application are build as you suggest, both of them would try to register the names for “/org” and for “org/example”. I expect this to cause a conflict: which application will receive the introspect request for /org/example? And what happen to the other?
    Instead each application should register its own unique path.

  2. regarding update of your repo, take your time, no worries. Plenty of other things to work on, so I won’t be sitting idling :stuck_out_tongue_winking_eye:

  3. d-feet is a tool to see and call dbus interface and methods. I thought the screeshot you posted earlier was taken from d-feet, because it looks very similar. I didn’t know about the qdbusviewer, thanks for mentioning it.

Hi Emanoil, thanks for the feedback. Here is my view on the various points. 1. registering each level of a path I think this is conceptually wrong as I mentioned in previous comments. Names on the bus have to be unique for dbus to work correctly. Now imagine two different applications providing two different services and interface:<br> /org/example/app1/interface1<br> /org/example/app2/interface2<br> If those application are build as you suggest, both of them would try to register the names for "/org" and for "org/example". I expect this to cause a conflict: which application will receive the introspect request for /org/example? And what happen to the other?<br> Instead each application should register its own *unique* path. 2. regarding update of your repo, take your time, no worries. Plenty of other things to work on, so I won't be sitting idling :stuck_out_tongue_winking_eye: 3. d-feet is a tool to see and call dbus interface and methods. I thought the screeshot you posted earlier was taken from d-feet, because it looks very similar. I didn't know about the qdbusviewer, thanks for mentioning it.
MicheleC commented 1 month ago
Owner

ok, I am probably wrong on point 1. As long as the paths of the applications are unique, it seems calls for the various path levels still works out fine, as you had actually posted in your previous comment :sweat_smile:

But I still don’t grasp what is the benefit in having to register each level of the path for the service. It seems to be just extra coding and extra work for not real benefit. We already know the path from the service xml file….

ok, I am probably wrong on point 1. As long as the paths of the applications are unique, it seems calls for the various path levels still works out fine, as you had actually posted in your previous comment :sweat_smile: But I still don't grasp what is the benefit in having to register each level of the path for the service. It seems to be just extra coding and extra work for not real benefit. We already know the path from the service xml file....
deloptes commented 1 month ago
Poster

Hi Michele,

thank you for looking into this much deeper. Here my comments

on 1. it took me most of the time until the writing of the example to understand how exactly this all works in dbus. I don’t know if my understanding is correct, but I assume so and here is my summary. When you register a service you have the service name, the path and the method. You register everything on a connection (which is identified by number and optionally you can give it a name like org.example.Service). So this is unique so far. Now here comes the path, where you can register and call the services interfaces and their methods.

To be able to browse through the path (and this was the original problem reported) you have to be able to return in the introspection of the path at each node. Because of how the generated code from dbusxml2qt3 is structured, it does not know the path at the level - there is no interface in this node and it can not reply. The node class itself does not know the path and it can handle only itself. For example if you register org.example.Service with path=/org/example/service - it listens only to /org/example/service and can not answer to introspecting “/”, or “/org” etc. IMO the easiest way is to let a node object listen to this part of the path and provide the necessary introspection.

In the use case where you would try registering two applications (lets say too services) on the same path if they use different connections it will be no issue, but if you try giving them the same name - the second will fail as name is unique.

If you try to register to the same connection - I don’t know what will happen. This would mean that in the code you should request the connection for the first service and try registering the second service on the same connection. But this would also mean that you would know where to register your service - skip the nodes or whatever.

I think this use case is worth testing and worth an example :)

So we have following options:

  connection      service/path       interface
1.    same             same              same          won't work
2.    same             same              diff          don't know
3.    same             diff              diff          will work
4.    diff             same              diff          don't know
5.    diff             diff              same          will work
6.    diff             diff              diff          will work
7.    diff             same              same          don't know
8.    same             diff              same          will work

1. because it is already listening
2. need to test
3. you can let different service listen on different path on same connection
4. need to test
5. obvious (for completeness)
6. obvious (for completeness)
7. need to test
8. same as 3

on 2. Luckily all is done and compiled, but unlikely that I finish this before the weekend. I follow the activities on TGW and I know very well that there is enough to do and you are not bored. Step by step we’ll find a solution hopefully sooner than later. It is good experience for me and we do something good for TDE at the end.

on 3. I installed it and indeed it does not show the service below. I’ll have a look at it as well. I did not know the tool - it is python written :|. I tested the example with qdbusviewer and dbus-send. Both return the element. We’ll find the truth at the end :)

Hi Michele, thank you for looking into this much deeper. Here my comments on 1. it took me most of the time until the writing of the example to understand how exactly this all works in dbus. I don't know if my understanding is correct, but I assume so and here is my summary. When you register a service you have the service name, the path and the method. You register everything on a connection (which is identified by number and optionally you can give it a name like org.example.Service). So this is unique so far. Now here comes the path, where you can register and call the services interfaces and their methods. To be able to browse through the path (and this was the original problem reported) you have to be able to return <node name=...> in the introspection of the path at each node. Because of how the generated code from dbusxml2qt3 is structured, it does not know the path at the level - there is no interface in this node and it can not reply. The node class itself does not know the path and it can handle only itself. For example if you register org.example.Service with path=/org/example/service - it listens only to /org/example/service and can not answer to introspecting "/", or "/org" etc. IMO the easiest way is to let a node object listen to this part of the path and provide the necessary introspection. In the use case where you would try registering two applications (lets say too services) on the same path if they use different connections it will be no issue, but if you try giving them the same name - the second will fail as name is unique. If you try to register to the same connection - I don't know what will happen. This would mean that in the code you should request the connection for the first service and try registering the second service on the same connection. But this would also mean that you would know where to register your service - skip the nodes or whatever. I think this use case is worth testing and worth an example :) So we have following options: connection service/path interface 1. same same same won't work 2. same same diff don't know 3. same diff diff will work 4. diff same diff don't know 5. diff diff same will work 6. diff diff diff will work 7. diff same same don't know 8. same diff same will work 1. because it is already listening 2. need to test 3. you can let different service listen on different path on same connection 4. need to test 5. obvious (for completeness) 6. obvious (for completeness) 7. need to test 8. same as 3 on 2. Luckily all is done and compiled, but unlikely that I finish this before the weekend. I follow the activities on TGW and I know very well that there is enough to do and you are not bored. Step by step we'll find a solution hopefully sooner than later. It is good experience for me and we do something good for TDE at the end. on 3. I installed it and indeed it does not show the service below. I'll have a look at it as well. I did not know the tool - it is python written :|. I tested the example with qdbusviewer and dbus-send. Both return the <node name=...> element. We'll find the truth at the end :)
MicheleC commented 1 month ago
Owner

Hi Emanoil,
yes, your understanding is correct and as long as two services are registered on different destinations, there should be no conflict. If we register two services on the same destination, the first one would use the name and the second would be queued and would be waiting for the name to become available once the first service terminates.

So your code would be ok, but as said I don’t see the benefit in doing all that and it is also not really adherent to the dbus specification.

1) from https://dbus.freedesktop.org/doc/dbus-specification.html#message-protocol-marshaling-object-path, you get

An object path is a name used to refer to an object instance. Conceptually, each participant in a D-Bus message exchange may have any number of object instances (think of C++ or Java objects) and each such instance will have a path.

So there should be a one-to-one correspondance between path and object(==service). But if we implement the code as per your suggestion, such correspondance would not be maintained since more than one path (/org, /org/example, /org/example/srvice) would be handled by the same service. And in case of multiple service registered on different paths, they would both be able to handle /org and /org/example introspection requests, although there would be no conflict since the destinations would be different.

2) why do we need to bother to be able to introspect each node of the path when we already know the service path? For example for a service registered on /org/example/service and providing org.example.service.interface, your code is already building all required levels in the code of the service. So what would be the extra functionality that we can achieve if dbusxml2qt3 was able to introspect all levels?

Hi Emanoil,<br> yes, your understanding is correct and as long as two services are registered on different destinations, there should be no conflict. If we register two services on the same destination, the first one would use the name and the second would be queued and would be waiting for the name to become available once the first service terminates. So your code would be ok, but as said I don't see the benefit in doing all that and it is also not really adherent to the dbus specification. 1) from https://dbus.freedesktop.org/doc/dbus-specification.html#message-protocol-marshaling-object-path, you get *An object path is a name used to refer to an object instance. Conceptually, each participant in a D-Bus message exchange may have any number of object instances (think of C++ or Java objects) and each such instance will have a path.* So there should be a one-to-one correspondance between path and object(==service). But if we implement the code as per your suggestion, such correspondance would not be maintained since more than one path (/org, /org/example, /org/example/srvice) would be handled by the same service. And in case of multiple service registered on different paths, they would both be able to handle /org and /org/example introspection requests, although there would be no conflict since the destinations would be different. 2) why do we need to bother to be able to introspect each node of the path when we already know the service path? For example for a service registered on /org/example/service and providing org.example.service.interface, your code is already building all required levels in the code of the service. So what would be the extra functionality that we can achieve if dbusxml2qt3 was able to introspect all levels?
deloptes commented 1 month ago
Poster

Hi Michele,

I first will answer a previous question. Why d-feet was not able to process example 4e.

The problem seems to be with the code generated for signals (in the example ObjectManager has 2 signals)

The code generated for the signals puts direction=“in” for some unknown reason.

methodElement = document.createElement("signal");
methodElement.setAttribute("name", "InterfacesAdded");

argumentElement = document.createElement("arg");
argumentElement.setAttribute("name",      "object");
argumentElement.setAttribute("type",      "o");
argumentElement.setAttribute("direction", "in");
methodElement.appendChild(argumentElement);

argumentElement = document.createElement("arg");
argumentElement.setAttribute("name",      "interfaces");
argumentElement.setAttribute("type",      "a{sa{sv}}");
argumentElement.setAttribute("direction", "in");
methodElement.appendChild(argumentElement);

as soon as you change direction=“out” it works with d-feet. It might be it is old bug. Even if I specify in the xml file direction=“out” it is writing “in” to the cpp file.

Hi Michele, I first will answer a previous question. Why d-feet was not able to process example 4e. The problem seems to be with the code generated for signals (in the example ObjectManager has 2 signals) The code generated for the signals puts direction="in" for some unknown reason. methodElement = document.createElement("signal"); methodElement.setAttribute("name", "InterfacesAdded"); argumentElement = document.createElement("arg"); argumentElement.setAttribute("name", "object"); argumentElement.setAttribute("type", "o"); argumentElement.setAttribute("direction", "in"); methodElement.appendChild(argumentElement); argumentElement = document.createElement("arg"); argumentElement.setAttribute("name", "interfaces"); argumentElement.setAttribute("type", "a{sa{sv}}"); argumentElement.setAttribute("direction", "in"); methodElement.appendChild(argumentElement); as soon as you change direction="out" it works with d-feet. It might be it is old bug. Even if I specify in the xml file direction="out" it is writing "in" to the cpp file.
deloptes commented 1 month ago
Poster

Hi Michele, now answering your question. I do not see any contradiction here. The service is register on the specified path “/org/example/service” and is working there as expected.

The problem is how the generated code handles the “node” - originally it was not handled at all or not properly. Now I am looking for a way to handle it properly. Thus we modify the code so that it could handle each node easily, by setting a node object that handles the introspection request. When you look at tdehw libdaemon it also handles each request, compares the path and returns the xml for this node. Except that it is difficult and ugly to read and write I do not see any difference how you get xml back. I did not think too much, but I guess the same questions might be asked for tdehw libdaemon - what would happen if I register service on the same path etc.

I understand your concern, so I will test like mentioned before and come back to you. Perhaps there might be a better solution, though I do not understand why this should be that bad. The advantage is that it is flexible and you can integrate services easily. I first tried to do something like in hwlib, but this changes the whole concept of current dbus-1-tqt. There must be one object listening on the connection and parsing each request to handle the correct path, which is very hard to change. The current solution just adds an object that handles one node and returns the next one in the introspection - same result as in hwlib.

But let me first update the code for the examples to match current dbus-1-tqt.

Meanwhile if times allow, could you have a look at methodgen.cpp line 341ff where it sets direction to “in” for the signals. It looks like we have found another bug - thanks to d-feet.

Thanks in advance

Hi Michele, now answering your question. I do not see any contradiction here. The service is register on the specified path "/org/example/service" and is working there as expected. The problem is how the generated code handles the "node" - originally it was not handled at all or not properly. Now I am looking for a way to handle it properly. Thus we modify the code so that it could handle each node easily, by setting a node object that handles the introspection request. When you look at tdehw libdaemon it also handles each request, compares the path and returns the xml for this node. Except that it is difficult and ugly to read and write I do not see any difference how you get xml back. I did not think too much, but I guess the same questions might be asked for tdehw libdaemon - what would happen if I register service on the same path etc. I understand your concern, so I will test like mentioned before and come back to you. Perhaps there might be a better solution, though I do not understand why this should be that bad. The advantage is that it is flexible and you can integrate services easily. I first tried to do something like in hwlib, but this changes the whole concept of current dbus-1-tqt. There must be one object listening on the connection and parsing each request to handle the correct path, which is very hard to change. The current solution just adds an object that handles one node and returns the next one in the introspection - same result as in hwlib. But let me first update the code for the examples to match current dbus-1-tqt. Meanwhile if times allow, could you have a look at methodgen.cpp line 341ff where it sets direction to "in" for the signals. It looks like we have found another bug - thanks to d-feet. Thanks in advance
MicheleC commented 1 month ago
Owner

Hi Emanoil,
let’s continue the discussion.

1) I can confirm the problem with the signal direction: once it is changed to “out”, the service shows up as it should in d-feet. I have created issue #19 as a reminder of that.

2) I have spent more time to check the behavior of other services as well and the ones that I have checked provide introspection for all node levels. Therefore even if for me it seems a waste of effort, I have to agree with you that we should provide the same functionality here. I should have done this comparison test earlier on, it would have saved a lot of discussions :sweat_smile:
I will look into the remaining part of the code of PR #8 in the coming days and merge it if ok or come back with comments if required.

Hi Emanoil,<br> let's continue the discussion. 1) I can confirm the problem with the signal direction: once it is changed to "out", the service shows up as it should in d-feet. I have created issue #19 as a reminder of that. 2) I have spent more time to check the behavior of other services as well and the ones that I have checked provide introspection for all node levels. Therefore even if for me it seems a waste of effort, I have to agree with you that we should provide the same functionality here. I should have done this comparison test earlier on, it would have saved a lot of discussions :sweat_smile:<br> I will look into the remaining part of the code of PR #8 in the coming days and merge it if ok or come back with comments if required.
deloptes commented 1 month ago
Poster

Thank you Michele and sorry that I am not able to do more at the moment. I was preparing the examples for update, but couldn’t make even this.

I’ll take a look at the use cases where it might be problematic just to be on the safe side, but perhaps I do it after you do your part. I already lost track of the problems but I think the amorphism of registerObject, the generation of dummy node and the async is outstanding + the new #19.

Let me know if I can somehow assist.

kind regards

Thank you Michele and sorry that I am not able to do more at the moment. I was preparing the examples for update, but couldn't make even this. I'll take a look at the use cases where it might be problematic just to be on the safe side, but perhaps I do it after you do your part. I already lost track of the problems but I think the amorphism of registerObject, the generation of dummy node and the async is outstanding + the new #19. Let me know if I can somehow assist. kind regards
MicheleC commented 1 month ago
Owner

Hi Emanoil, thanks for updating your repo, that will save some time during testing :+1:

Hi Emanoil, thanks for updating your repo, that will save some time during testing :+1:
Sign in to join this conversation.
No Milestone
No Assignees
3 Participants
Due Date

No due date set.

Dependencies

This issue currently doesn't have any dependencies.

Loading…
Cancel
Save
There is no content yet.