我试图将Tensorflow的官方基本word2vec实现转换为使用tf.Estimator.
问题是当使用Tensorflow Estimators时,丢失函数(sampled_softmax_loss或nce_loss)会出错.它在原始实现中完美地运行.
这是Tensorflow的官方基本word2vec实现：
以下是我实施此代码的Google Colab笔记本,该代码正常运行.
https://colab.research.google.com/drive/1nTX77dRBHmXx6PEF5pmYpkIVxj_TqT5I
这是Google Colab笔记本,我在其中更改了代码,因此它使用Tensorflow Estimator,它不起作用.
https://colab.research.google.com/drive/1IVDqGwMx6BK5-Bgrw190jqHU6tt3ZR3e
为方便起见,这里是我定义model_fn的Estimator版本的精确代码
batch_size = 128
embedding_size = 128 # Dimension of the embedding vector.
skip_window = 1 # How many words to consider left and right.
num_skips = 2 # How many times to reuse an input to generate a label.
num_sampled = 64 # Number of negative examples to sample.
def my_model( features,labels,mode,params):
with tf.name_scope('inputs'):
train_inputs = features
train_labels = labels
with tf.name_scope('embeddings'):
embeddings = tf.Variable(
tf.random_uniform([vocabulary_size,embedding_size],-1.0,1.0))
embed = tf.nn.embedding_lookup(embeddings,train_inputs)
with tf.name_scope('weights'):
nce_weights = tf.Variable(
tf.truncated_normal(
[vocabulary_size,stddev=1.0 / math.sqrt(embedding_size)))
with tf.name_scope('biases'):
nce_biases = tf.Variable(tf.zeros([vocabulary_size]))
with tf.name_scope('loss'):
loss = tf.reduce_mean(
tf.nn.nce_loss(
weights=nce_weights,biases=nce_biases,labels=train_labels,inputs=embed,num_sampled=num_sampled,num_classes=vocabulary_size))
tf.summary.scalar('loss',loss)
if mode == "train":
with tf.name_scope('optimizer'):
optimizer = tf.train.GradientDescentOptimizer(1.0).minimize(loss)
return tf.estimator.EstimatorSpec(mode,loss=loss,train_op=optimizer)
这里是我称之为估算和培训的地方
word2vecEstimator = tf.estimator.Estimator(
model_fn=my_model,params={
'batch_size': 16,'embedding_size': 10,'num_inputs': 3,'num_sampled': 128,'batch_size': 16
})
word2vecEstimator.train(
input_fn=generate_batch,steps=10)
INFO:tensorflow:Calling model_fn.
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
编辑：根据请求,这是input_fn的典型输出
print(generate_batch(batch_size = 8,num_skips = 2,skip_window = 1))
(array([3081,3081,12,6,195,195],dtype=int32),array([[5234],[ 12],[ 6],[3081],[ 195],[ 2]],dtype=int32))
最佳答案
原文链接：https://www.f2er.com/python/438936.html