[netopeer-server] transapi development: not a single callback found for the configuration change

[hongmat]

Hello,

I have developed many transAPI modules in the past, but recently I have encountered a really annoying problem that I cannot solve.

Each time I enable my transAPI, netopeer-server displays:
Model "proxy-conf" transAPI: there was not a single callback found for the configuration change.

My callbacks are correctly defined in proxy-conf.c.
I am able to parse a file proxy.conf when the transAPI is loaded, and to append the content of the file inside a xml tree buffer (for further get/edit). I can use edit-config on the running datastore, but the modifications must also be applied to proxy.conf which only works IF the callbacks are called.

I have regenerated hundred times my transAPI with lnctools and reinstalled netopeer-server from scratch. Nothing worked.

I'm using transapi version 6 with libnetconf 0.10.

I urgently need some help.
Thanks.

[michalvasko]

Hi,
your proxy-conf.c likely has something wrong in it, but it should be enough if you provide the paths file that you used to generate it, the schema, and the output of netopeer-server -v3.

Regards,
Michal

[hongmat]

Here is a snapshot of my datastore.xml:

<?xml version="1.0" encoding="UTF-8"?>
<datastores xmlns="urn:cesnet:tmc:datastores:file">
  <running lock="">
    <proxyconf xmlns="urn:proxy:com:proxyconf">
      <node1>3129</node1>
      <node2>512 MB</node2>
      <node3>lru</node3>
      <node4>lru</node4>
      <node5>0</node5>
      <node6>0 KB</node6>
      <node7>4096 KB</node7>
      <node8>90</node8>
      <node9>95</node9>
      <node10>
        <peerIp>127.0.0.1</peerIp>
        <peerRule>127.0.0.1 sibling 3128 3130 proxy-only no-digest no-netdb-exchange</peerRule>
      </node10>
    </proxyconf>
  </running>
  <startup lock="" modified="true" />
  <candidate lock="" modified="true" />
</datastores>

Here is the struct definition inside proxyconf.c:

struct transapi_data_callbacks clbks =  {
	.callbacks_count = 12,
	.data = NULL,
	.callbacks = {
		{.path = "/proxy:proxyconf/proxy:node1", .func = callback_proxy_proxyconf_proxy_node1},
                {.path = "/proxy:proxyconf/proxy:node2", .func = callback_proxy_proxyconf_proxy_node2},
                {.path = "/proxy:proxyconf/proxy:node3", .func = callback_proxy_proxyconf_proxy_node3},
                {.path = "/proxy:proxyconf/proxy:node4", .func = callback_proxy_proxyconf_proxy_node4},
                {.path = "/proxy:proxyconf/proxy:node5", .func = callback_proxy_proxyconf_proxy_node5},
                {.path = "/proxy:proxyconf/proxy:node6", .func = callback_proxy_proxyconf_proxy_node6},
                {.path = "/proxy:proxyconf/proxy:node7", .func = callback_proxy_proxyconf_proxy_node7},
                {.path = "/proxy:proxyconf/proxy:node8", .func = callback_proxy_proxyconf_proxy_node8},
                {.path = "/proxy:proxyconf/proxy:node9", .func = callback_proxy_proxyconf_proxy_node9},
                {.path = "/proxy:proxyconf/proxy:node10", .func = callback_proxy_proxyconf_proxy_node10},
		{.path = "/proxy:proxyconf/proxy:node10/proxy:peerIp", .func = callback_proxy_proxyconf_proxy_node10_proxy_peerIp},
		{.path = "/proxy:proxyconf/proxy:cachePeer/proxy:peerRule", .func = callback_proxy_proxyconf_proxy_cachePeer_proxy_peerRule}
	}
};

[hongmat]

Hi Michal,

Here is the file paths_file:

ietf-inet-types=urn:ietf:params:xml:ns:yang:ietf-inet-types
proxy-conf=urn:proxy:com:proxyconf

/proxy:proxyconf/proxy:node1
/proxy:proxyconf/proxy:node2
/proxy:proxyconf/proxy:node3
/proxy:proxyconf/proxy:node4
/proxy:proxyconf/proxy:node5
/proxy:proxyconf/proxy:node6
/proxy:proxyconf/proxy:node7
/proxy:proxyconf/proxy:node8
/proxy:proxyconf/proxy:node9
/proxy:proxyconf/proxy:node10
/proxy:proxyconf/proxy:node10/proxy:peerIp
/proxy:proxyconf/proxy:node10/proxy:peerRule

Here is the yang data model:

module proxy-conf {

    namespace "urn:proxy:com:proxyconf";

    prefix "proxy";

    import ietf-inet-types { prefix "inet"; }

    organization
       "zzz";

    contact
       "xxx <xxxxx@yyyy.com>";

    description
       "YANG version of the netconf controller for Custom Proxy.

   revision "2015-03-27"{
       description
           "Datastore model for Custom Proxy configuration.";
   }
   
   typedef percent{
       type uint16{
           range "0 .. 100";
       }
       description "Percentage";
   }

   container proxyconf{
       description
           "Configuration and operational parameters for Custom Proxy.";
      
       leaf node1{
           type inet:port-number;
           must 'current() <= 10000' {
               error-message
                   "Proxy port number out of bound";
           }
           default 3129;
       }
      
       leaf node2{
           type string;
           default "256 MB";
       }
      
       leaf node3{
           type string;
           default "lru";
       }
      
       leaf node4{
           type string;
           default "lru";
       }
      
       leaf node5{
           type uint32;
           default 0;
       }
      
       leaf node6{
           type string;
           default "0 KB";
       }
      
       leaf node7{
           type string;
           default "4096 KB";
       }
      
       leaf node8{
           type percent;
           default 90;
       }
      
       leaf node9{
           type percent;
           default 95;
       }
       
       container node10{
           description
          	"List of cache peer rules.";
           
           leaf peerIp{
                mandatory true;
           	type string;
      	   }
	   
           leaf peerRule{
                mandatory true;
           	type string;
      	   }
       }
   }
}

[michalvasko]

Hi,
if you are changing /proxyconf/node10/peerRule, then the path is modified in your proxyconf.c file. Otherwise it seems fine, if this is not the problem, then please provide netopeer server verbose output like I asked.

Regards,
Michal

[hongmat]

Here is netopeer-server -v 5 output at startup:

netopeer-server[13874]: Datastore proxy-conf initiated with ID 1681692778.
netopeer-server[13874]: 
***[v1.0-1] Parsing Proxy configuration file...
netopeer-server[13874]:  node1 = 3125
netopeer-server[13874]:  node2 = 512 MB
netopeer-server[13874]:  node3 = lru
netopeer-server[13874]:  node4 = lru
netopeer-server[13874]:  node5 = 0
netopeer-server[13874]:  node6 = 50 KB
netopeer-server[13874]:  node7 = 3 GB
netopeer-server[13874]:  node8 = 80
netopeer-server[13874]:  node9 = 95
netopeer-server[13874]: 
[node10]
netopeer-server[13874]: hostname = 127.0.0.1
netopeer-server[13874]: type = sibling
netopeer-server[13874]: proxy_port = 3128
netopeer-server[13874]: icp_port = 3130
netopeer-server[13874]: general_option = proxy-only
netopeer-server[13874]: reverse_proxy_option1 = no-digest
netopeer-server[13874]: reverse_proxy_option2 = no-netdb-exchange


netopeer-server[13874]: proxy-conf initialized.
netopeer-server[13874]: Model "proxy-conf" transAPI: there was not a single callback found for the configuration change.
netopeer-server[13874]: Starting FMON thread for proxy-conf data model.
netopeer-server[13874]: Netopeer server successfully initialized.

[hongmat]

Even if I simplify the data model to have only 10 nodes (no child for node10), the problem remains the same: not a single callback can be found :(

[hongmat]

I never encountered such a problem with transapi version 5.

[michalvasko]

Hm, perhaps the generated file is wrong because the paths file has invalid namespace prefix? You defined proxy-conf, but used proxy.

[hongmat]

I don't think it comes from the ns. I have changed prefix and ns on the fly while posting here, but in my own files, everything's consistent.

[michalvasko]

Hi,
you must post exactly the files you are using, which are compiled and loaded by netopeer, how else am I supposed to help you if you keep changing things and claiming the problems I see do not exist in your local files? So please, fix all those files, compile it and try again. If the result is similar, post it all again unmodified, thank you.

Regards,
Michal

[hongmat]

Code, prefix etc. displays my company name. which I want to avoid to divulge.
I'm gonna double check and regenerate .so & .la. I'll let you know in few minutes.

[hongmat]

Hi Michal,

I have slightly modified the yang data model. Here is the new one:

module np-proxy-conf {

    namespace "urn:np:com:proxyconf";

    prefix "nppc";

    import ietf-inet-types { prefix "inet"; }

    organization
       "None";

    contact
       "hongmat";

    description
       "YANG model of the netconf controller for remote proxy configuration.";

   revision "2015-03-27"{
       description
           "Datastore model for proxy configuration.";
   }
   
   typedef percent{
       type uint16{
           range "0 .. 100";
       }
       description "Percentage";
   }

   container proxyconf{
       description
           "Proxy configuration and operational parameters.";
      
       leaf node1{
           type inet:port-number;
           must 'current() <= 10000' {
               error-message
                   "Port number out of bound";
           }
           default 3128;
       }
      
       leaf node2{
           type string;
           default "256 MB";
       }
      
       leaf node3{
           type string;
           default "lru";
       }
      
       leaf node4{
           type string;
           default "lru";
       }
      
       leaf node5{
           type uint32;
           default 0;
       }
      
       leaf node6{
           type string;
           default "0 KB";
       }
      
       leaf node7{
           type string;
           default "4096 KB";
       }
      
       leaf node8{
           type percent;
           default 90;
       }
      
       leaf node9{
           type percent;
           default 95;
       }
       
       list cachePeer{
           key "peerIp";
           
           leaf peerIp{
              type string;
           }

           leaf peerRule{
              type string;
           }
       }
   }
}

Here is the paths_file:

ietf-inet-types=urn:ietf:params:xml:ns:yang:ietf-inet-types
np-proxy-conf=urn:np:com:proxyconf

/nppc:proxyconf/nppc:node1
/nppc:proxyconf/nppc:node2
/nppc:proxyconf/nppc:node3
/nppc:proxyconf/nppc:node4
/nppc:proxyconf/nppc:node5
/nppc:proxyconf/nppc:node6
/nppc:proxyconf/nppc:node7
/nppc:proxyconf/nppc:node8
/nppc:proxyconf/nppc:node9
/nppc:proxyconf/nppc:cachePeer
/nppc:proxyconf/nppc:cachePeer/nppc:peerIp
/nppc:proxyconf/nppc:cachePeer/nppc:peerRule

Here is the output at startup (netopeer-server -v 5):

netopeer-server[27502]: Transapi calling callback /A:netopeer/A:modules/A:module/A:enabled with op ADD.
netopeer-server[27502]: Adding transapi "/usr/local/etc/netopeer/cfg-proxy-conf/np-proxy-conf.yin"
netopeer-server[27502]: ncds_features_parse: no feature definitions found in data model np-proxy-conf.
netopeer-server[27502]: ncds_new_internal: Relax NG validator set (/usr/local/etc/netopeer/np-proxy-conf/np-proxy-conf-config.rng)
netopeer-server[27502]: ncds_new_internal: Schematron validator set (/usr/local/etc/netopeer/np-proxy-conf/np-proxy-conf-schematron.xsl)
netopeer-server[27502]: Datastore np-proxy-conf initiated with ID 1681692778.
netopeer-server[27502]: 
***[v1.0-3] Parsing proxy configuration file...
netopeer-server[27502]: node1 = 3125
netopeer-server[27502]: node1  = 512 MB
netopeer-server[27502]: node3  = lru
netopeer-server[27502]: node4  = lru
netopeer-server[27502]: node5  = 0
netopeer-server[27502]: node6  = 50 KB
netopeer-server[27502]: node7 = 3 GB
netopeer-server[27502]: node8 = 80
netopeer-server[27502]: node9 = 95
netopeer-server[27502]: np-proxy-conf initialized.
netopeer-server[27502]: Model "np-proxy-conf" transAPI: there was not a single callback found for the configuration change.
netopeer-server[27502]: Starting FMON thread for np-proxy-conf data model.
netopeer-server[27502]: Netopeer server successfully initialized.

I have also created a function called by transapi_init() which parses each line of the proxy configuration and builds the xml tree in memory. The resulting output is as follows:

node type: Element, name: node1, content: 3125
node type: Element, name: node2, content: 512 MB
node type: Element, name: node3, content: lru
node type: Element, name: node4, content: lru
node type: Element, name: node5, content: 0
node type: Element, name: node6, content: 50 KB
node type: Element, name: node7, content: 3 GB
node type: Element, name: node8, content: 80
node type: Element, name: node9, content: 95

node type: Element, name: cachePeer, content: 127.0.0.1127.0.0.1 sibling 3128 3130 proxy-only no-digest no-netdb-exchange

node type: Element, name: cachePeer, content: 192.168.254.23192.168.254.23 parent 3128 3130 proxy-only no-digest no-netdb-exchange

To summarize:
The xml tree is correctly dumped after transAPI initialization.
I am able to perform get-config and get-config --filter commands.
As for edit-config, the changes are made inside the datastore, but no changes are made in the proxy configuration file, which is normal as the server displays "there was not a single callback found for the configuration change".

Best,
Hongmat

[michalvasko]

Hi,
the paths file has the same problem as before, namespace prefix defined as np-proxy-conf, but the one used is nppc. Also, since you are not using any paths from ietf-inet-types, you do not need to define this namespace. The namespace definition is completely independent and local only to the paths file, simply change np-proxy-conf to nppc and hopefully it will work.

Regards,
Michal

[hongmat]

Hi,
It fixed the problem, thank you very much Michal :-)
And thanks for the info about the namespace definition.

Best,
Hongmat