Tensorflow: Confusion regarding the adam optimizer : <p>I'm confused regarding as to how the adam optimizer actually works in tensorflow.</p>

<p>The way I read the <a href="https://www.tensorflow.org/versions/r0.9/api_docs/python/train.html#AdamOptimizer" rel="noreferrer">docs</a>, it says that the learning rate is changed every gradient descent iteration. </p>

<p>But when I call the function I give it a learning rate. And I don't call the function to let's say, do one epoch (implicitly calling # iterations so as to go through my data training). I call the function for each batch explicitly like</p>

<pre><code>for epoch in epochs
     for batch in data
          sess.run(train_adam_step, feed_dict={eta:1e-3})
</code></pre>

<p>So my eta cannot be changing. And I'm not passing a time variable in. Or is this some sort of generator type thing where upon session creation <code>t</code> is incremented each time I call the optimizer?</p>

<p>Assuming it is some generator type thing and the learning rate is being invisibly reduced: How could I get to run the adam optimizer without decaying the learning rate? It seems to me like <a href="https://www.tensorflow.org/versions/r0.9/api_docs/python/train.html#RMSPropOptimizer" rel="noreferrer">RMSProp</a> is basically the same, the only thing I'd have to do to make it equal (learning rate disregarded) is to change the hyperparameters <code>momentum</code> and <code>decay</code> to match <code>beta1</code> and <code>beta2</code> respectively. Is that correct?</p>
