在他们的github issues上进行了一些痛苦的挖掘后,似乎需要一个tf.cond来正确使用它并且还有一个’resue = True’标志,以便BN移位和缩放变量被正确地重复使用.在弄清楚之后,我提供了一个小小的描述,说明我认为是使用它的正确方法here.

现在我已经编写了一个简短的脚本来测试它(只有一个层和一个ReLu,很难让它比这个小).但是,我不是百分百确定如何测试它.现在我的代码运行时没有错误消息,但意外返回NaN.这降低了我对我在其他帖子中提供的代码可能正确的信心.或许我所拥有的网络很奇怪.无论哪种方式,有人知道什么是错的？这是代码：

import tensorflow as tf
# download and install the MNIST data automatically
from tensorflow.examples.tutorials.mnist import input_data
from tensorflow.contrib.layers.python.layers import batch_norm as batch_norm

def batch_norm_layer(x,train_phase,scope_bn):
    bn_train = batch_norm(x,decay=0.999,center=True,scale=True,is_training=True,reuse=None,# is this right?
    trainable=True,scope=scope_bn)

    bn_inference = batch_norm(x,is_training=False,reuse=True,scope=scope_bn)

    z = tf.cond(train_phase,lambda: bn_train,lambda: bn_inference)
    return z

def get_NN_layer(x,input_dim,output_dim,scope,train_phase):
    with tf.name_scope(scope+'vars'):
        W = tf.Variable(tf.truncated_normal(shape=[input_dim,output_dim],mean=0.0,stddev=0.1))
        b = tf.Variable(tf.constant(0.1,shape=[output_dim]))
    with tf.name_scope(scope+'Z'):
        z = tf.matmul(x,W) + b
    with tf.name_scope(scope+'BN'):
        if train_phase is not None:
            z = batch_norm_layer(z,scope+'BN_unit')
    with tf.name_scope(scope+'A'):
        a = tf.nn.relu(z) # (M x D1) = (M x D) * (D x D1)
    return a

mnist = input_data.read_data_sets("MNIST_data/",one_hot=True)
# placeholder for data
x = tf.placeholder(tf.float32,[None,784])
# placeholder that turns BN during training or off during inference
train_phase = tf.placeholder(tf.bool,name='phase_train')
# variables for parameters
hiden_units = 25
layer1 = get_NN_layer(x,input_dim=784,output_dim=hiden_units,scope='layer1',train_phase=train_phase)
# create model
W_final = tf.Variable(tf.truncated_normal(shape=[hiden_units,10],stddev=0.1))
b_final = tf.Variable(tf.constant(0.1,shape=[10]))
y = tf.nn.softmax(tf.matmul(layer1,W_final) + b_final)

### training
y_ = tf.placeholder(tf.float32,10])
cross_entropy = tf.reduce_mean( -tf.reduce_sum(y_ * tf.log(y),reduction_indices=[1]) )
train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)
with tf.Session() as sess:
    sess.run(tf.initialize_all_variables())
    steps = 3000
    for iter_step in xrange(steps):
        #Feed_dict_batch = get_batch_Feed(X_train,Y_train,M,phase_train)
        batch_xs,batch_ys = mnist.train.next_batch(100)
        # Collect model statistics
        if iter_step%1000 == 0:
            batch_xstrain,batch_xstrain = batch_xs,batch_ys #simualtes train data
            batch_xcv,batch_ycv = mnist.test.next_batch(5000) #simualtes CV data
            batch_xtest,batch_ytest = mnist.test.next_batch(5000) #simualtes test data
            # do inference
            train_error = sess.run(fetches=cross_entropy,Feed_dict={x: batch_xs,y_:batch_ys,train_phase: False})
            cv_error = sess.run(fetches=cross_entropy,Feed_dict={x: batch_xcv,y_:batch_ycv,train_phase: False})
            test_error = sess.run(fetches=cross_entropy,Feed_dict={x: batch_xtest,y_:batch_ytest,train_phase: False})

            def do_stuff_with_errors(*args):
                print args
            do_stuff_with_errors(train_error,cv_error,test_error)
        # Run Train Step
        sess.run(fetches=train_step,train_phase: True})
    # list of booleans indicating correct predictions
    correct_prediction = tf.equal(tf.argmax(y,1),tf.argmax(y_,1))
    # accuracy
    accuracy = tf.reduce_mean(tf.cast(correct_prediction,tf.float32))
    print(sess.run(accuracy,Feed_dict={x: mnist.test.images,y_: mnist.test.labels,train_phase: False}))

当我运行它时,我得到：

Extracting MNIST_data/train-images-idx3-ubyte.gz
Extracting MNIST_data/train-labels-idx1-ubyte.gz
Extracting MNIST_data/t10k-images-idx3-ubyte.gz
Extracting MNIST_data/t10k-labels-idx1-ubyte.gz
(2.3474066,2.3498712,2.3461707)
(0.49414295,0.88536006,0.91152304)
(0.51632041,0.393666,nan)
0.9296

它曾经是最后一个是南,现在只有少数几个.一切都好还是我是偏执狂？