@Ricky – 11K x 4090 x 8 = ~343 MB (not 200 MB as you said). And this memory needs to be contiguous, since its a regular numeric matrix. Even if you have this much memory available, it is quite possible that the largest contiguous block of memory is smaller than 343 MB. If you need to do this for 2 other arrays of the same size, you only make things worse…

I was reading through your posts about memory management and pre-allocation of arrays and found them very interesting. Currently I am stuck in a very peculiar situation. My input data file is a .mat file of size 52 MB. I load the file in MATLAB using the load command. I scan through the file and extract my data points out of this. I get 11000 rows and each row has 4090 columns. I preallocate a large array using zeros command. I know my data type is double and it uses 8 bytes. Total bytes used for above array is 200 MB. For further processing of my data I have to allocate 2 more arrays of the same previous rows and columns.

So while doing this I run out of memory how can I free up the previous memory allocated and how can I solve this problem.

(I have to use 2007b because the symbolic toolbox changed and I only get errors now with newer versions 🙁 ).

@Yair, thanks for the link.

@Naor – sorry, I don’t know for sure. I assume that at least part of the warm-up time is indeed related to the data size, because the data needs to be allocated in memory and then loaded onto the CPU cache. Once it’s in the cache, data access times will be much faster.

Other warm-up factors include function compilation time (which is more-or-less constant), disk caching for the relevant m-files, and Matlab engine being swapped-in from virtual memory and other similar aspects that are dynamic in nature. If there’s any I/O involved in the function, then I/O caching may also come into play here.

Former MathWorker Bill McKeeman says in his Performance Tuning document that he usually throws the initial 3 measurements away when profiling – read his document, it’s illuminating.

The JIT discussion reminds me of a very important question I have had for many years regarding Matlab’s function “warm-up” phenomenon. Everyone knows, I’m sure, that the first time you run a function in Matlab it takes a lot longer than the next run through. There are some obvious reasons for this and quite likely other not obvious reasons. But we all know to “warm up” a function before benchmarking it. The question I have not been able to answer is do we need to worry about warm-up before our production runs on large datasets too. In other words, is the first-run slow-down simply constant overhead or does it grow with problem size is some way? If so, what constitutes a “first” run? First time in a Matlab session? After a clear all command? Something else?

I guess it’s not very directly related to this post, but I thought Yair might know something about this.

Thanks,
-n