Multi GPU Training in Tensorflow (Data Parallelism) when Using feed_dict -

i use multiple gpus train tensorflow model taking advantage of data parallelism.

i training tensorflow model using following approach:

x_ = tf.placeholder(...) y_ = tf.placeholder(...) y = model(x_) loss = tf.losses.sparse_softmax_cross_entropy(labels=y_, logits=y) optimizer = tf.train.adamoptimizer() train_op = tf.contrib.training.create_train_op(loss, optimizer) in epochs:    b in data:       _ = sess.run(train_op, feed_dict={x_: b.x, y_: b.y}) 

i take advantage of multiple gpus train model in data parallelize manner. i.e. split batches in half , run each half batch on 1 of 2 gpus.

cifar10_multi_gpu_train seems provide example of creating loss draws graphs running on multiple gpus, haven't found examples of doing style of training when using feed_dict , placeholder opposed data loader queue.

update

seems like: https://timsainb.github.io/multi-gpu-vae-gan-in-tensorflow.html might provide example. seem pull in average_gradients cifar10_multi_gpu_train.py , create 1 placeholder slice each of gpus. think need split create_train_op 3 stages: compute_gradients, average_gradients , apply_gradients.