c# - Simultaneously read from one file and write to another -


i want read bytes file, process them, , write process's output out file. since reading/writing bottleneck (the processing quick) should able double speed (if files on different drives) simultaneously reading , writing.

i can inter-process-communication, seems overkill. there simple way achieve this?

edit

maybe wasn't clear enough. want following:

  1. read chunk
  2. process it
  3. start writing output
  4. go 1 while writing

edit 2

@downvoters care explain why? that obvious dasblinkenlight wrote it's not worthy of question? doubt i'm only 1 didn't know that.

i should able double speed simultaneously reading , writing.

unless read whole file in memory @ once, doubling speed unlikely, because os optimizes reading latency through readahead.

os applies principle of locality of reference: figures out program reading sequentially, , uses read-ahead prefetch next few blocks. time program done writing , ready reading again, data in buffer, os serve program. in time program spends waiting write finish, os prefectch more data, cycle continue virtually no wait on reading side.

as matter of experiment try making own program use asynchronous i/o employing readasync , writeasync apis. process go follows:

  1. before entering loop, set task pendingwrite = null
  2. start read readasync , await completion
  3. if no additional data available, exit loop (step 8)
  4. process data needed
  5. see if there pending write task. if there is, await completion
  6. initiate next write writeasync, , assign task pendingwrite
  7. go step 2
  8. await completion of pendingwrite task last cycle.

Comments