Sharing Parameters between BucketingModules #10987
Replies: 6 comments
-
It seems that other people are having the very same problem: https://discuss.mxnet.io/t/sharing-parameters-between-two-modules-through-arg-dict/1046. |
Beta Was this translation helpful? Give feedback.
-
You can hack module by yourself. |
Beta Was this translation helpful? Give feedback.
-
@chinakook Could you please elaborate on that? I assume you mean deriving a new class from |
Beta Was this translation helpful? Give feedback.
-
you can refer to this hacking mod = mx.mod.Module(sym, data_names=['data'], label_names=['softmax_label'], context=ctx)
mod.bind(data_shapes=[...]
, label_shapes=[...])
mod.init_params(initializer=mx.init.Xavier(magnitude=2.))
mod.init_optimizer(optimizer='sgd', optimizer_params={'learning_rate':0.01, 'momentum': 0.9}) bind a new symbol mod._symbol = new_sym
mod.binded=False
mod.bind(data_shapes=[...]
, label_shapes=[...], shared_module=mod) You also need to hack parameters that the symbols does not share. |
Beta Was this translation helpful? Give feedback.
-
@chinakook Excuse me, I do not see how this solves my problem; |
Beta Was this translation helpful? Give feedback.
-
@sandeep-krishnamurthy Could you please label it as question |
Beta Was this translation helpful? Give feedback.
-
Description
I am trying to share parameters between two BucketingModules. More specifically, my requirements are:
BucketingModule
. The modules have some parameters in common, and they should share them. I.e. the parameter arrays should essentially be identical.What is your recommendation? How should I set up those modules?
Things I've tried
shared_module
attribute:https://mxnet.incubator.apache.org/api/python/module/module.html?highlight=bucketing#mxnet.module.BucketingModule.bind
Unfortunately, this is not available for
BucketingModule
. (Why?)https://gist.github.com/bricksdont/3dc6213482c38526decacf2a54b7f8b6
But the downside is obviously that it slows down training.
Environment info (Required)
I'm using the Python package.
Beta Was this translation helpful? Give feedback.
All reactions