IsolatedStorage Performance problem

02-24-2009 12:26 PM |

I think I found a performance problem with IsolatedStorage. It seems that the time to write a file depends HEAVILY on how many bytes are written to it at a time. The runtime seems linear in the number of WRITE_CHUNKs written! In particular, I open a file of 10MB from the local hard disk using OpenFileDialog, reading "READ_CHUNK" number of bytes at a time, and writing "WRITE_CHUNK" number of bytes at a time to the IsolatedStorage (storage size increased to 99MB already), and the runtime is:

(1) READ_CHUNK = 1 MB, WRITE_CHUNK = 1MB number of write = 10 runtime = 1 seconds
(2) READ_CHUNK = 1 MB, WRITE_CHUNK = 10K number of write = 1000 runtime = 25 seconds
(3) READ_CHUNK = 10K, WRITE_CHUNK = 10K number of write = 1000 runtime = 25 seconds
(4) READ_CHUNK = 1 MB, WRITE_CHUNK = 1K number of write = 10000 runtime = 4 minutes

The runtime varies a lot although the same total number of bytes are written to IsolatedStorage. Also, the runtime is insensitive to the READ_CHUNK size (comparing (2) and (3)). So this is not a general problem with reading from/writing to stream, but specific to IsolatedStorage. This looks like a performance bug to me... perhaps each write to IsolatedStorage incurs a full scan of the bytes already written?

For the longest time I thought IsolatedStorage was unusable due to its poor performance but it now seems it is a (fixable) performance bug. For the time being the workaround is to use a large buffer and to minimize the number of writes. Feedback is welcomed. Also cross-posted to the bug forum.

The code is as follows:

void save_to_isolated(System.IO.Stream stream, string name) {
System.IO.IsolatedStorage.IsolatedStorageFile store =
System.IO.IsolatedStorage.IsolatedStorageFile.GetUserStoreForApplication();
if (!store.DirectoryExists("debug"))
store.CreateDirectory("debug");
System.IO.IsolatedStorage.IsolatedStorageFileStream file_stream =
store.CreateFile("debug/" + name);
if (file_stream == null)
throw new Exception();

int READ_CHUNK = 1024 * 1024;
int WRITE_CHUNK = 1 * 1024;
byte[] buffer = new byte[READ_CHUNK];
while (true) {
int read = stream.Read(buffer, 0, READ_CHUNK);
if (read <= 0) break;
int to_write = read;
while (to_write > 0) {
file_stream.Write(buffer, 0, Math.Min(to_write, WRITE_CHUNK));
to_write -= Math.Min(to_write, WRITE_CHUNK);
}
}
iestream.Close();
file_stream.Close();
}

Re: IsolatedStorage Performance problem

02-28-2009 2:50 AM |

Very nice tip.

You probably will get even better performance if the code is multithreaded.  Did you try it on different speed of machines?  (i.e. a 3.5 GigHz single core with an older HD vs a dual core with a SATA drive).  I wonder if the larger buffers might actually slow down on older HD models (?)

Re: IsolatedStorage Performance problem

02-28-2009 11:00 AM |

It was confirmed by Microsoft that it was a performance issue with the way it has to check the validity and size of the IsolatedStorage on every write (and will hopefully be addressed in SL3).  The slow runtime has nothing to do with the speed of the hard drive, since reading from local hard drive using OpenFileDialog doesn't have this problem.  Also, in my experiment, the run was done on a background thread.  To make such a simple operation (which supposedly should take <1s) a multithreaded one sounds excessive to me.

Re: IsolatedStorage Performance problem

07-22-2009 5:06 PM |

Hi, Seems is not fixed on 3.0 version, any idea when will be fixed? It's terrible slow writing in a IsolatedStorage comparing with FileStream on Windows platform.

Re: IsolatedStorage Performance problem

07-22-2009 5:35 PM |

Well, the workaround is simple though, which is to use a buffered stream.  You write to the buffered stream, and the buffered stream writes to IsolateStorage in big chunks.

Re: IsolatedStorage Performance problem

07-23-2009 5:42 PM |

For certain reasons I have to Seek a lot in Stream and because of this I cannot implement easy a buffered solution. Imagine I write at position 2000 then I have to write some bytes at position 20 and Seek again to 4000 -just an example. How can I handle this? Thx!

Re: IsolatedStorage Performance problem

07-23-2009 5:51 PM |

Do you know the exact size of the stream that you are working on?  If so, you can simply create a byte[] array, create a memory stream out of it, and work on the memory stream.  When you are done, write the memory stream to the IsolatedStorage.

I won't recommend random access the IsolatedStorage at all.  Besides from the performance issue, it is also error prone...  i.e. long-running IO operation is more error-prone.  Better to think of reading and writing to/from IsolatedStorage as an atomic operation.