python - Reading multiple images, process them to one image and feed through model -


is there way build following graph in tensorflow:

  1. load n images (n can vary each set) using tf queues , tf image readers.
  2. process these images fixed size image , prepare batches.
  3. feed these batches through cnn model

some questions/info:

  1. i trying build data loading part in tf instead of python functions , feed_dict. guess, tf data loading can train model faster compared python , feed_dict. right ?
  2. building graph small n (n<5) easy. define exclusive nodes each image in n , process on them. (working)
  3. can use tf "while_loop" build such functionality read n images ??
  4. does keras supports such functionality ?

thanks suggestions.

i did last week! awesome, learned ton tensorflow using things tf.map_fn, , tf.cond. , worked.

this week refactored code eliminate all, because bad idea.

issues ran into:

  1. doing preprocessing in tensorflow messy debug. doing proper tdd benefit here, still not going particularly pretty or easy debug.
  2. you should offloading preprocessing cpu , leaving gpu (assuming you're using one) training. better approach have queue , load thread/class that's dedicated preprocessing task. , doing work in numpy/scikit/scikit-image going easier configure , test.
  3. i thought smart, corralling code single model. complexity of preprocessing meant model hard iterate on, got rigid code - example when added test set evaluation in, preprocessing requirement different. had add large sections of conditional code model , got ugly quick.

that being said, preprocessing steps maybe more complex yours. if you're sticking simple things can apply of simple image preprocessing steps might still easier go approach.

to answer questions specifically:

  1. queues won't give benefit on feed_dict know of. still have problem of moving data tf queue on cpu gpu memory each iteration same feed_dict does, watch thread if care topic, gpu queues coming: https://github.com/tensorflow/tensorflow/issues/7679

  2. you should dequeue_many queue, process them batch. if need each individual image use tf.map_fn remove first dimension , pass individual 3d images specified function. heed warning above when go route - you'll happier doing in separate thread.

  3. already answered in #2, use tf.map_fn iterate on multiple images in batch. it's pretty easy use actually.

  4. i don't know keras.


Comments