What does it mean when MOSEK reports out of space?

MOSEK requires a certain amount of memory to store a given optimization problem. The exact amount is proportional to the number of variables, constraints and non-zero entries in the problem. Moreover, some additional memory is required to optimize it. How much depends on the structural properties of the problem. In general it is not possible to predict how much memory MOSEK will require to solve a particular problem.

Depending on the operating system different options exist to circumvent the out of space problem.

On 32bit Windows (and other 32bit systems) each process can normally only allocate up to 2GB memory, even if more physical memory is available. Please read Microsoft's explanation for details. Starting from MOSEK version 3.1 then it is possible to make MOSEK use 3GB of RAM in 32 bit Windows if you make some operating system settings described in Microsoft's explanation. Installing more physical memory may be a solution up to the 2GB or 3GB limit. If the requirement goes beyond that it may be necessary to switch to a 64bit platform.
On 64bit operating systems the per-process memory limit is significantly higher, and the limitation is usually the amount of physical memory installed on the machine.